It started with a phony bill.
During the COVID-19 pandemic, much of University of Vermont Health Network’s nonclinical workforce shifted to remote work. An employee’s daughter received an email she believed was from her homeowners association. When she couldn’t open the attachment, she forwarded it to her mother, who opened it on her work computer and then later connected to the health system’s VPN.
Just like that, the organization was exposed to ransomware.
“You don’t have the opportunity to see it happening across systems and try to stop it,” says CISO Nate Couture. “They just all go down. We had 1,300 servers go offline in a 15-minute span.”
Incidents like the October 2020 ransomware attack against one of Vermont’s largest health systems force IT leaders to confront the difference between cyber recovery and disaster recovery. Traditional DR plans assume that organizations can quickly restore systems from backups and resume operations. However, cyber incidents require a fundamentally different approach, with healthcare organizations sometimes using temporary systems for weeks as they rebuild their environments.
“Traditional disaster recovery means getting back to normal from physical and environmental incidents,” says Lee Kim, senior principal for cybersecurity and privacy at HIMSS. “Cyber incident recovery means getting back to normal from cybersecurity incidents. These are intangible things that normally are not within our line of sight unless we look for them. When we find out about them, it is often at a later stage, after data may be exfiltrated or other damage is done.”
Click the banner below to read the recent CDW Cybersecurity Research Report.
Delivering Care Amid an IT Crisis
The attack on University of Vermont Health Network exposed a critical — and extremely common — gap in preparedness strategy. While the health system had downtime procedures for routine IT issues, those plans were designed to accommodate outages lasting only a few hours, or maybe days. But with the organization’s Epic electronic health record system offline for four weeks, paper-based workarounds quickly proved inadequate.
“There are certain clinical services where there is just no such thing as a downtime plan,” Couture says. “You can do patient charting on paper, and you can do a lot of your regular ambulatory care and emergency room care without a lot of technology. But you cannot do radiation oncology without the technology that supports it.”
For a typical disaster recovery incident, the organization would have simply shut down radiation oncology treatment until systems were back online, bumping a few appointments in the process. But with systems down for weeks, that would have represented an unacceptable gap in care, and so Couture and his team built out an isolated interim environment that allowed cancer treatments to continue, even as IT teams raced to complete forensic analysis and restoration. They also stood up an offline version of the health system’s EHR, connected directly to some desktop computers and printers, and used the setup to deliver printouts of patient data to clinicians.
LEARN MORE: Observability improves IT system performance and supports patient care.
Beyond restoring systems, the hospital had to replace 5,500 compromised endpoints and implement new security tools in the middle of the crisis. The response included deploying CrowdStrike’s Falcon EDR platform, migrating to Rubrik’s immutable backups and partnering with Zscaler for cloud-based security visibility when on-premises tools failed.
The new solutions, Couture says, put the health system in a better position to prevent a repeat attack — and to recover more quickly if one occurs.
“We’re in a fight where you’re never going to get to punch back,” he says. “Your job is to try to not get hit, to be able to take a punch if you do get hit and to stand yourself back up off the mat when you get knocked down.”
Preparing for Extended Downtime Procedures
Since 2018, Memorial Hermann Health System in Houston has periodically conducted ransomware exercises to evaluate how the organization would continue to provide care for patients if an attack brought systems down for an extended period.
“The first time we did this, we brought in our executive leadership teams and representatives from clinical operations,” says CISO Randy Yates. “We asked what would happen if we experienced an attack and they couldn’t use certain IT systems: ‘What would happen if this went away, if you couldn’t print, if you couldn’t scan?’”
Yates and his team felt it was necessary to build out cyber resilience tools and processes that would help clinical and operations teams keep doing their jobs, even if the network went down for days or weeks.
DISCOVER: Healthcare organizations need a cyber resilience strategy that supports success.
Practice Makes Perfect for Cyber Resilience
Adam Lee, director for emergency management and organizational resilience at Memorial Hermann, led a two-year effort to map critical processes across departments, identifying what each area needed to maintain operations for 30 days rather than just a few hours.
The health system also began conducting technical red team/blue team exercises, where third parties attempt controlled attacks to test monitoring capabilities. These exercises helped clarify what the organization could do to better support cyber resilience after an attack.
“Sometimes, it’s making better use of the tools that we have,” Yates says. “But we realized that, in some cases, we needed new capabilities.”
