Risks and Challenges of Using AI in Pediatrics
Large language models are prone to “sycophancy,” or the desire to satisfy the user prompting it, as well as hallucinations, which involve fabricated findings, Desai explains.
He notes that using AI in pediatrics also presents challenges because many children’s diseases are rare and therefore more difficult for algorithms to predict.
“This is a pure biostatistical limitation: The performance of an algorithm in real life depends heavily on the prevalence of the condition. The more common the condition, the more accurate the algorithm,” Desai explains. “Conversely, the rarer the condition, the worse the algorithm will perform.”
Although scientists use machine learning algorithms to identify rare pediatric conditions such as primary immune deficiencies, they still generate a large amount of “fake positive” results, according to Desai. Children then participate in lots of unnecessary and expensive testing, he says.
READ MORE: AI is being used in healthcare in several different ways today.
“That makes these algorithms very difficult to implement at the point of care,” Desai says.
In addition, training biases bring risks in pediatrics.
“Some groups of patients were perhaps underrepresented in the training data, so the model’s predictions about those patients are inaccurate,” Desai says.
To manage the risks of AI, Texas Children’s has established an AI governance and steering committee, says Teresa Tonthat, the hospital’s vice president and associate CIO.
The governance committee ensures all AI model outcomes require a “human in the middle” to verify information before making patient decisions, Tonthat says. The committee also addresses concerns around regulatory requirements, bias and AI hallucinations, she adds.
Because the health system is working with data about children, Texas Children’s provides education to care teams about signoffs through Epic’s MyChart and works with vendors such as Microsoft to discuss how they leverage and protect patient data, Tonthat says.
“Our risk tolerance is very low when it comes to information about our pediatric patients,” Tonthat says.
Despite the risks, AI brings opportunities for solving clinical issues and improving efficiency in workflows.
AI for Radiology and Diagnostics in Pediatrics
Texas Children’s has been using AI for more than decade and has turned to predictive modeling, automation, deep learning and machine learning to solve clinical problems, Tonthat says. The hospital has been exploring generative AI for the past two years, particularly with models that improve care team workflows. Interest took time to build among the clinical teams at Texas Children’s, but now the staff is ready to scale up, she says.
Texas Children’s created an AI model for radiologists to predict bone age. The model was trained on millions of X-rays of pediatric hands.
“Because we have millions of X-rays of hand images, we have trained the model to let them know within seconds what the age of that child’s hand is from a bone-density perspective,” Tonthat says.
Using its bone age AI-based prediction model, Texas Children’s was able to improve turnaround time by 50% by integrating AI into radiologists’ clinical workflows, according to Tonthat. The AI model is a collaboration between the hospital’s radiology and information services departments and, its AI governance and guidance committee.
CHOP is also using AI in radiology, according to Desai.
“In research, investigators at CHOP are looking at AI for improved radiology diagnostics, intercepting lab errors, accelerating diagnosis from pathology slides and more,” he says.
If a doctor is seeing a patient with poorly controlled asthma, an ambient AI tool could automatically retrieve the patient’s asthma history, summarize past asthma-related visits and notify a doctor if the patient is at higher risk for influenza, Desai says. It then would check to see which medication the patient’s insurance plan covers and could start composing the order for the asthma controller, Desai says.
