How AR & VR in Healthcare Enhances Medical Training

Johns Hopkins Creates Adaptable VR for Medical Training

The Johns Hopkins School of Nursing has implemented VR training at all levels, from doctoral to prelicensure nursing, according to Kristen Brown, assistant professor at the Johns Hopkins School of Nursing and the simulation strategic projects lead at the Johns Hopkins Medicine Simulation Center. Areas of focus for training include resuscitation, anaphylactic reactions, postsurgical management, acute-care management and pediatric critical care.

Brown is working on a multiplayer VR study involving medical and nursing trainees with practice scenarios that accommodate up to 100 learners.

The VR setup at Johns Hopkins comprises an Oculus headset and an Alienware computer from Dell Technologies. Although the VR configuration includes hand haptics, cognitive decision-making is more critical to VR medical training, according to Brown. She wanted artificial intelligence functionality that could adapt to what students were learning.

“We make decisions based on what’s going on — time-critical decisions,” Brown says. “So one of the important components was that there was some sort of AI that’s really adapting to what we’re doing.”

READ MORE: Dr. José M. Barral discusses using virtual tools for anatomy education.

Orthopedic Students Practice VR Surgical Training at Mass General

Dr. Augustus D. Mazzocca, chief of the division of sports medicine at Massachusetts General Hospital, medical director at Mass General Brigham Sports Medicine and faculty member at Harvard Medical School, uses a system called PrecisionOS to augment orthopedic education. Mass General’s Department of Orthopedic Surgery also is beginning to use PrecisionOS to train medical students and residents in motor skills required for surgery.

The virtual reality headsets create a realistic environment for the student to practice. However, there is no real-time feedback since it’s an individual experience, says Mazzocca. One of the advantages of the PrecisionOS system is that the educator can connect the VR glasses to a laptop device, allowing the professor to see exactly what the trainee is seeing and doing. This gives insight and helps the educator do a better job of instructing.

“The beauty of virtual reality is now I can see what the learner is looking at for as long as they’re looking at it, and I think that really helps teaching,” Mazzocca says.

VR also lets students practice an exercise multiple times compared with work done in a cadaver or bio skills lab, where students can do exercises only once or twice, Mazzocca says. Now professors can let students know if they’re not looking at an X-ray long enough or adjusting their hands correctly, he says. Three-dimensional ability in surgery is important, and training to adjust a hand to place a pin or device in multiple planes is critical.

“With the ability to look and practice and see things in 3D, it’s really spectacular,” Mazzocca says.

Key aspects of orthopedic VR surgery include learning the correct steps for a procedure and shooting an X-ray at the right angle, according to Mazzocca. Students also learn how to perform a reduction of a fractured bone, which involves setting a broken bone through pushing and pulling so the ends line up correctly. VR lets students practice knee or shoulder replacement or pinning fractures many times without having to practice on live patients, he explains.

Mazzocca considers the field of VR to be at the “ground floor.” He says VR is not mainstream yet, but he sees room for growth to include all areas of orthopedic surgery, including hip fractures, ACL reconstructions and shoulder instability operations.

VR in medicine will “continue to grow and be better, but right now, I think it’s terrific for student and resident education,” Mazzocca says.

AR Medical Training Provides a Graphic Look Inside the Body at UCF

The University of Central Florida College of Nursing implemented AR and VR for nurses and nurse practitioners over the past year, says Mindi Anderson, interim associate dean for simulation and immersive learning and director of the school’s healthcare simulation graduate program. 

One of the benefits of using AR and VR comes from a look inside the anatomy of the human body. “There are scenarios where the student can walk inside the body and see what’s happening behind the scenes with that pathophysiology, anatomy and physiology, so they can really get that understanding of what is happening with the patient. This can help with interventions,” Anderson says.

Students can see, for example, what happens during a stroke or heart attack, she says. One way UCF has used AR is to prepare students to participate in a manikin-based simulation with similar content, Anderson says.

With AR, users can see the environment around them. “I could see the floor, but now I see a holographic patient in a bed on that floor as opposed to virtual reality, where everything is blocked out from your vision,” Anderson explains.

EXPLORE: How virtual reality and extended reality are advancing patient care.

UCF uses both Microsoft HoloLens and Oculus headsets, as well as a variety of prepackaged simulations. The school chooses cases in which real-life simulations would be challenging, including maternity complications such as postpartum hemorrhage. In maternal care, an image can be overlayed on a manikin’s abdomen, Anderson says.

“In AR, they can see a normal birth, and they could see a birth that didn’t go right,” Anderson says. “This allows learners the opportunity to recognize something that is not normal and intervene earlier, thus, leading to improved patient outcomes.”

Best Practices for Implementing AR and VR in Healthcare

Brown offers the following suggestions when using AR and VR for medical training:

Use multiplayer scenarios. Train together with teams of nurses and physicians, rather than in a silo. “You don’t work in your bubble; you work with a team,” Brown says. “Where you have a single player, that’s fine, but in the real world, you’re not working as a single player, you’re working as a collaborative team.”

Prioritize practice. Take advantage of the ability to practice repeatedly in AR and VR.

“You want to get that learner to feel a little bit stressed so they can then practice over and over again,” Brown says, “So when the stressful situation happens, they can manage it, they’ve seen it before, and they feel more comfortable. We were able to give our learners an opportunity to practice and to make decisions in real time and repeat it, which was really important.”