Ambient Listening in Healthcare: Dictation, Documentation and AI

What Is Ambient Listening, and How Is It Used In Healthcare?

Ambient listening is voice recognition technology that uses AI to listen to, interpret and analyze conversations between patients and providers. Many vendors operate in this space, including Microsoft’s Nuance, Amazon Web Services, Suki, eClinicalWorks’ Sunoh.ai, DeepScribe, Abridge, Ambience and Augmedix.

Critically, these tools aren’t general-purpose listening tools. “Microsoft’s AI is specifically designed and adapted for each healthcare use case — in this example, documentation and workflow automation,” says Kenneth Harper, general manager of Dragon at Microsoft. “We tailor the technology to the needs and challenges of healthcare.”

Ambient listening goes beyond generating a transcript — the key feature of traditional dictation services — to create clinically accurate summaries, generate billing and diagnostic codes, and capture information to draw up orders for labs, prescriptions, follow-up visits or other procedures.

DISCOVER: Enhance existing investments with workflow optimization.

Automating routine tasks can offer numerous benefits, letting clinicians focus their attention on patient care. It alleviates the frustration of doing administrative work and improves the accuracy and level of detail in documentation, which Soni says can help practices receive full reimbursement for the care they provide.

For many providers, the time savings presents an opportunity to reduce “pajama time,” or the time spent outside the clinic on transcribing and documenting in-clinic visits. For others, however, ambient listening can contribute to revenue gains. “Some clinicians choose to use the time savings they get to see more patients,” Soni notes.

Harper says the technology also has value as a recruitment tool. “Healthcare leaders can use ambient listening to demonstrate that they care not only about the patient but also about helping their clinicians reclaim the joy of practicing medicine.”

How Do Clinicians Feel about Ambient Listening?

By and large, clinical users — including but not limited to physicians, nurses, physician assistants, residents and medical students — look favorably on ambient listening technology.

A New England Journal of Medicine Catalyst paper published in February from the Permanente Medical Groups found a largely favorable response to the use of an unnamed ambient AI scribe after a two-month pilot across 21 sites in Northern California. Physicians spent less time writing notes, both in the office and after hours, and anecdotally reported being better able to listen to patients during appointments. Roughly 28% of physicians enabled the AI scribe in more than 100 patient encounters; one physician used it in 1,210 encounters.

The results of a similar (albeit smaller) pilot at Stanford Medicine paint a similar picture. Two-thirds of 48 physician users said ambient listening saved time, 78% said it expedited note taking, and 96% said it was easy to use. Given these results, the health system said it plans to roll out the application — in this case, Dragon Ambient eXperience (DAX) Copilot from Microsoft Nuance — to all providers.

EXPLORE: How should healthcare organizations navigate the workforce crisis?

How Is Ambient Listening Enhanced by EHR Integrations?

The NEJM Catalyst paper noted the primary barriers to adopting ambient listening — such as the steps to get started and the difficulty of accessing the tool — were typical of any new technology in healthcare.

The primary solution to these problems, the authors said, is electronic health record integration. “A major goal is to directly integrate AI scribe tools into the EHR,” they wrote, “so that barriers to ease of access and use are eliminated, and the documentation can be more seamlessly blended into existing workflows.”

“Integrations allow us to keep chart content between Suki and the EHR in sync,” Soni says. “This gives users incredible flexibility in how they document.” A clinician could document within the EHR — completing charts before a visit, for example — and then sync it with an ambient listening tool. Or, the clinician could start documentation in Suki, pull relevant data such as vital signs from the EHR, and then send the finished note to the EHR, where relevant information gets updated.

Soni says another important feature of EHR integration is the ability to use clinical data from patient encounters and improve the AI models operating behind the scenes: “We can fine-tune the output of the models to be as relevant as possible.”

DIVE DEEPER: How does EHR optimization improve clinical workflows?

What’s Next for Ambient Listening Technology?

Integrating ambient listening with clinical systems beyond the EHR illustrates the technology’s potential. One example is the ability to listen to alerts from sensors and other monitoring devices in patients’ rooms and provide context so clinicians can make more informed decisions. Another is the potential to search clinical data for early signs of a disease, especially rare conditions that are often difficult to diagnose.

Meanwhile, Stanford Medicine physicians who oversaw the health system’s pilot pointed to the “transformative impact” of advances in ambient listening, such as editing drafts in real time using natural language processing and customizing notes to an individual provider’s style.

Soni says he sees ambient listening evolving to take on “any tasks that can be given to a human assistant” and “fully freeing up clinicians to focus on taking care of their patients.”

Microsoft’s Harper also says the future for ambient listening goes “far beyond just note taking” and automating administrative tasks to “serving up intelligent information that enables clinicians to make better-informed decisions and helping to support clinicians in delivering personalized patient care.”