From Years of Uncertainty to Answers: How AI Is Shortening the Rare Disease Diagnostic Odyssey
Key Takeaways:
- ThinkRare, an algorithm at the Children’s Hospital of Eastern Ontario, scans health records to flag children who may need a rare disease referral. It has led to 21 new diagnoses and a 70% success rate so far.
- At Boston Children’s Hospital, a reasoning model built by OpenAI solved all 20 known test cases and contributed to 18 diagnoses among 376 previously unresolved ones.
- Neither tool works alone. Data bias, hallucinations, and clinicians’ tendency to defer to AI mean a human in the loop remains essential.
Why a rare disease diagnosis can take half a decade
For people living with a rare disease, the route to a diagnosis can be long, circuitous, and occasionally never completed at all. Reaching an answer often demands specialist training and technical resources that many clinics simply do not have. In a News and Perspectives article for JMIR, correspondent Simon Spichak reports on how AI initiatives at one children’s hospital in the United States and another in Canada are working to close that gap, and how they could fundamentally reshape the diagnostic experience for children and families living with rare conditions.
For some people, the benefits are already tangible. AI algorithms have helped their doctors reach a diagnosis, and that diagnosis has in turn opened the door to accessibility accommodations, treatments, clinical trials, and further research into their condition. If these tools can be scaled, they could meaningfully shorten what is often described as the diagnostic odyssey – the almost five years, on average, of appointments, medical visits, and frustration that it typically takes to diagnose a rare disease.
Finding the children who are lost in the system
CHEO is helping to lead the way with ThinkRare, an algorithm that scans electronic health records and flags children who may need a referral to a rare disease specialist for further testing.
“The purpose of ThinkRare is to find the patients that aren’t coming to us, that are bouncing around, that are lost in the system,” says Ivan Terekhov, BCom, director of research informatics, AI, and technology at CHEO Research Institute.
The approach targets the very beginning of the diagnostic journey, where children are cycling through appointments without anyone joining the dots. ThinkRare has already prompted genetic sequencing and follow-up in a handful of children, leading to 21 new rare disease diagnoses and a 70% success rate to date. The researchers behind the algorithm are now working on rolling the model out to other hospitals.
Tackling the cases that leave clinicians stuck
BCH is addressing the opposite end of the odyssey, using a reasoning model built by OpenAI to help resolve cases that have defeated clinicians.
Catherine Brownstein, PhD, scientific director of the genetic investigations arm of the Manton Center for Orphan Disease Research at BCH, says she was sceptical at first. When the team fed the reasoning model rare disease cases for which the diagnosis was already known, it initially appeared to falter. Eventually it diagnosed 19 out of 20 cases correctly, and ultimately all 20.
“When we looked at the one that it got wrong,” she says, “we were wrong.” The AI had flagged a second diagnosis that clinicians had missed.
At that point, Brownstein and her team judged the model ready for the real test: helping to solve cases that remained undiagnosed. It proved remarkably successful, with its output leading to 18 diagnoses among 376 previously unresolved cases.
What does AI do that clinicians cannot?
According to a recent report, insufficient physician training may help to explain why people with rare diseases go undiagnosed for so long. ThinkRare addresses this gap by incorporating expert-curated criteria to flag rare diseases that the average doctor might miss, so that those children can be considered for genetic testing.
Even after sequencing, rare disease specialists can take a long time to arrive at a diagnosis, and complicated cases sometimes remain unsolved. Specialists have only a limited amount of time to spend with each family, and may be carrying many cases they are simply unable to revisit.
During her PhD, Brownstein spent six years working on a single family. “There’s just simply not enough geneticists in this world or scientists in this world to do the thorough, in-depth investigations that are necessary to make all the insights possible,” she says.
The reasoning model used at BCH, designed to reason carefully and show its work, was developed using o3 Deep Research, a publicly accessible model. When it identifies a potential genetic mutation that might be causing a child’s symptoms, it presents its chain of reasoning alongside the suggestion. Geneticists then take over, running the tests needed to confirm or rule out the diagnosis.
“But instead of having to go to the sequence and find that variant yourself, it’s presented to you in a way that’s much, much quicker,” says Brownstein.
The insights the model produces may also lead to discoveries that clinicians would not otherwise make. In one instance it surfaced a new genetic variant that might be implicated in vitiligo, an autoimmune condition in which the body attacks the pigment-producing cells in the skin.
“I was speechless,” Brownstein recalls. Her team now has research under way to investigate whether the mutation is causative. If that work bears out, she hopes it could point towards a new treatment.
As tools like these move closer to everyday practice, the demand for clinicians who understand both their promise and their failure modes is rising. Short, focused CPD training such as the College of Contemporary Health’s AI Essentials for Primary Care course is designed for exactly that: helping practitioners get to grips with how these systems work, where they add value, and where human judgement has to stay firmly in charge.
Why there needs to be a human in the loop
Neither model removes the clinician from the process.
ThinkRare’s algorithm is passive, flagging a small number of cases out of several hundred thousand children attending its clinics. Those children still need testing and assessment by a specialist, because the algorithm does not actually predict which condition a child may have.
And because the algorithm is built on clinical expertise and trained on electronic medical records, bias is baked in. “I think we’d be naive to say that there’s not data quality issues or there’s worse data quality for those who are not White and those who do come from rural locations,” notes Alexandre White-Brown, MSc, a genetic counsellor and project manager of ThinkRare. Canada’s diversity is nonetheless well represented within the algorithm’s training dataset, which helps to offset some of these potential issues.
While Brownstein says the reasoning model used at BCH improved rapidly, errors and hallucinations are common with any large language model (LLM). OpenAI’s reasoning model still needs to be used in collaboration with a clinician rather than as a stand-alone tool.
The risk of trusting the machine too readily
Research into human-AI interaction has repeatedly shown that experts may defer to the judgement of an AI system, even when it is wrong.
“Previous studies show that learning from evidence is especially difficult when the evidence is poorer and uncertainty is higher, as in the case of patients with rare diseases,” says Aranzazu Viñas, PhD, an assistant professor at the University of the Basque Country. “Therefore, AI’s errors might be still more difficult to detect in the case of rare diseases.”
Her recent study suggests that doctors will trust an AI’s judgement even when they receive new clinical information showing it to be wrong. Viñas emphasises the importance of conducting “ecological research” to establish whether these errors also appear in real-world settings. Beyond that, she says it is “important to develop strategies and protocols that increase human critical thinking and detection of AI errors.”
The future of AI for rare diseases
BCH is now looking towards further study and model validation, measuring time and cost savings, clinician effort, false positives, and whether using the model actually changes the care people receive. The team is working with OpenAI to democratise access for other researchers and clinicians.
At CHEO, ThinkRare is currently able to spot the low-hanging fruit, but Terekhov points to the data silos created by information locked away in unstructured reports and doctors’ notes. The next iteration of ThinkRare will incorporate LLMs that can access and interpret this material, allowing the system to flag more cases. Scaling the model countrywide will require regulators to sign it off.
What a diagnosis changes for a family
White-Brown explains the impact ThinkRare is already having. Antony was diagnosed with an ultrarare condition called Chung-Jansen Syndrome at 10 years old, after being flagged by the algorithm. The diagnosis provided an explanation for his symptoms, and testing revealed that the same condition affected his brothers and his mother.
“The biggest thing for this family was actually being able to access resources in school that were previously unattainable,” says White-Brown.
CCH insight
AI-assisted triage, diagnostic support, and documentation tools are arriving in clinical settings faster than most training has kept pace with. The College of Contemporary Health’s AI Essentials for Primary Care CPD short course gives healthcare professionals a practical grounding in how these technologies work, how to appraise their output critically, and how to use them safely alongside clinical judgement.
Explore AI Essentials for Primary Care →
Source: JMIR Publications
Read More