Spatial Computing: What Do Healthcare IT Leaders Need to Know?

What Is Spatial Computing?

Spatial computing combines physical and digital realities into one space. Rather than reading information on a two-dimensional screen — a phone or laptop, for example — spatial computing technology allows people to process and interact with data in a 3D format. Virtual, augmented and mixed reality are all forms of spatial computing.

How Is Spatial Computing Used in Healthcare?

Developers have built hundreds of apps for the Apple Vision Pro, and providers are exploring their best uses in clinical and educational settings.

Medical Education

Virtual and augmented reality platforms are increasingly used to supplement medical training. Immersive simulations allow doctors, nurses and other providers to not only practice procedures but also experience the strong emotions that critical care situations can trigger. 

“A student can, for example, have a visceral sense of what a code blue situation feels like,” Lichtenstein explains.

“There are psychological and emotional results of spatial computing that, if leveraged correctly, can help people learn in a way they couldn’t with any other technology,” says Dr. Brennan Spiegel, director of health services research for Cedars-Sinai in Los Angeles and a 2023 HealthTech influencer. “If you think about the strongest memories from your own life, they’re usually tied to an emotion. People learn best from experiences they can imagine in their heads.” 

EXPLORE: Virtual reality helps healthcare students experience clinical scenarios.

Mental and Behavioral Health

The Centers for Disease Control and Prevention estimates that 20 percent of U.S. adults have a mental illness, and workforce data shows that more than half of Americans live in areas with a shortage of mental health providers. Researchers are exploring how spatial computing can be used with artificial intelligence to expand access to mental and behavioral health services and help providers treat more patients. 

One innovation is the therapist-trained AI platform that Spiegel co-founded: the eXtended-Reality Artificially Intelligent Ally, or Xaia, which runs on the Apple Vision Pro. Patients speak to a digital robot that also generates contextually relevant virtual environments. For instance, Xaia may create a rock garden or play soothing music for a patient who describes feeling anxious. 

“Xaia can augment the care that’s already being provided by a human,” says Spiegel, adding that Xaia alerts the therapist if it detects a risk of self-harm. His team is currently teaching the system to write clinical notes.

Enhanced Imaging

Lichtenstein, who helped launch Sharp HealthCare’s Spatial Computing Center of Excellence to explore new ways to use Apple Vision Pro in healthcare settings, describes the device as a “phenomenal tool for people who need to touch anatomy in some form.” He explains that new spatial apps are providing 3D views of anatomical images, which radiologists can use for more precise diagnoses. 

He also predicts that the technology will become more widely used as a presurgical tool. “We’re seeing a high level of interest from our surgeons and proceduralists about this,” Lichtenstein says. “The technology can help them understand the nuances of a patient’s specific anatomy before going into the operating room.” 

Faster Delivery of Information 

Rather than having data spread over multiple screens, a spatial computing device can collect all of the information in one space. Stanford Medicine recently deployed the Apple Vision Pro in this manner during a heart procedure. The doctor used a virtual monitor to zoom in on each piece of data he needed without having to look away from the patient.

“The device constantly renders all the information on the periphery of the clinician’s point of view, and they use just their eyes to bring the data they need front and center,” explains Gostine, who spoke with the Stanford team about the procedure.

In addition to its clinical uses, Gostine says, this type of condensed data visualization can help hospital operations become more efficient. “Right now, hospitals build costly command centers with people interpreting data from hundreds of screens. That could be migrated to smaller, distributed command centers where spatial computing technology constantly surfaces information that needs to be acted on,” he says.

More Dynamic Health Records

Spatial computing technology is being used to enhance the ways that clinicians access and interact with medical records. Epic, which is partnering with Sharp HealthCare on spatial computing efforts, recently launched the first electronic health record app for the Apple Vision Pro. 

“Right now, we pull the information into a virtual reality space,” Lichtenstein explains. “Down the road, maybe we can use augmented reality to superimpose data in the real world.” 

LEARN MORE: Is your healthcare organization ready for virtual reality?

What Should Hospitals Know Before Deploying Spatial Computing?

“Finding the right use cases for spatial computing technology is the key to its success,” Gostine says. “Like everything in healthcare, if you force the right technology into the wrong clinical environment, it becomes an expensive tool that people don’t use.”

He continues, “My advice for health systems would be to take a long-term approach. Make these tools available to clinicians to experiment with, find the areas where it’s working and slowly expand out from there.”

“We have an amazing opportunity to do things with education, anatomy and the preclinical phase of care,” Lichtenstein adds. “I think we’re only at the beginning of what will be a longer story about how we’re using both augmented and virtual reality to deliver care to patients.”