Collaboration and Preparation: What Mixed Reality Surgery Affords the Operating Room

Microsoft’s HoloLens offered a glimpse of how surgeons across the world could help each other and share information.

Your browser doesn’t support HTML5 audio

Mixed reality and image-guided technologies now offer surgeons new preoperative planning tools, and a means to collaborate and share expertise across the globe.

A recent 24-hour surgery event facilitated by Microsoft featured 12 holographic surgeries, along with roundtables and live interviews on the topic. A dozen doctors from around the world performed multiple surgeries, including a shoulder replacement in South Africa and a knee procedure in the United Arab Emirates.

The surgeons demonstrated how mixed reality solutions enable them to collaborate across borders, access data, and improve accuracy by layering information over the patient as a guide for procedures.

Microsoft’s HoloLens headsets are mixed reality smart glasses that uses sensors, advanced optics and holographic processing to seamlessly blend 3D imagery and physical environments. Experts say the advancement of mixed reality surgery offers many benefits, including improved health outcomes, more accurate surgical procedures and better training worldwide.

For example, in December 2020, a surgeon in the U.S. and a surgeon in France used mixed reality technology to assist a surgeon in Brazil who needed to repair a collarbone fracture and perform a shoulder arthroscopy.

Dr. John Erickson, an orthopedic hand and upper extremity surgeon with Atlantic Health System in New Jersey, and Dr. Thomas Grégory, chief of the department of orthopedic and trauma surgery at Avicenne Hospital in Paris, used a HoloLens 2 mixed reality headset to help Dr. Bruno Gobbato in Jaraguá do Sul, Brazil. Using Microsoft applications and Teams, the surgeons could see the patient and holographic images, helping Gobbato perform the procedure.

Eventually, you could have your entire operating team wearing devices and sharing an environment where even your techs and residents can see the same model you are using.”

Dr. John Erickson

orthopedic hand and upper extremity surgeon, Atlantic Health System

“They were my partners helping me with the surgery,” Gobbato said in a Microsoft news release last month. “We had a French perspective, we had an American perspective, and we had a Latin American perspective. We had one-quarter of the world inside the operating room."

‘Scratching the Surface’ with Mixed Reality Surgery

As the HoloLens enables surgeons to collaborate with and help other surgeons globally, Erickson says another primary benefit of mixed reality is in preoperative planning. Software now allows surgeons to create immersive 3D models of anatomical parts to better prepare for procedures, which can result in shorter operating times, less blood loss and less risk. A 2017 study by Stanford Medicine found that mixed reality could improve the accuracy of women’s oncology surgery and minimize the need for more invasive mastectomies.

“It’s valuable to surgeons to go through the surgery ahead of time, and it leads to better outcomes for patients because you are mentally checked in, have planned the surgery and know what to expect," Erickson tells HealthTech.

The ability to share information and point-of-view insight will better train the next generation of surgeons, and improve the skills and knowledge of operating room staff, he says.

“Eventually, you could have your entire operating team wearing devices and sharing an environment where even your techs and residents can see the same model you are using,” Erickson adds. “We are really just scratching the surface.”

More On Virtual Reality, Training, Patient-Centered Care, Augmented Reality,