AI Model Helps Clinicians Tell Epilepsy Apart from Functional Seizures Using Routine Clinical Notes
Key Takeaways:
- Researchers developed “EpiScreen”, an AI system built on fine-tuned large language models, which distinguished epilepsy from psychogenic non-epileptic seizures (PNES) using only routine clinical notes, achieving an AUC of up to 0.992 within one institution and 0.980 across institutions.
- When clinicians were given EpiScreen’s predictions and sentence-level explanations, neurology residents’ diagnostic accuracy rose from 76.3%–79.3% to 87.7%–89.0%, and senior attending physicians’ accuracy rose from 81.7% to 89.3%.
- The authors propose EpiScreen as a decision-support tool used after neurology referral and before confirmatory video-electroencephalography (vEEG), though prospective real-world trials are still needed.
An AI approach to a long-standing diagnostic challenge
In a recent study published in the journal npj Digital Medicine, researchers describe the development and validation of “EpiScreen”, a novel artificial intelligence (AI) system designed to distinguish epilepsy from psychogenic non-epileptic seizures (PNES) using clinical notes from routinely documented electronic health records (EHRs).
The retrospective, two-institution study drew on 13,633 patient records. EpiScreen outperformed comparison models, achieving an area under the receiver operating characteristic curve (AUC) of up to 0.992 in within-institution testing and 0.980 in cross-institutional testing. AUC measures a model’s ability to distinguish between the two conditions across different classification thresholds. In a retrospective case-review assessment, EpiScreen also improved clinicians’ classification accuracy.
Why telling the two conditions apart matters
Epilepsy is a chronic neurological disorder characterised by recurring, unprovoked seizures caused by sudden, abnormal electrical discharges in the brain. Approximately 50 million people live with the condition worldwide.
Diagnosis can be complicated by PNES, also termed functional or dissociative seizures. These are paroxysmal events (sudden, temporary seizure-like episodes) that resemble epileptic seizures but are not caused by epileptic electrical activity. Because the two conditions have different underlying mechanisms and management needs, accurate differentiation is critical. Misdiagnosis remains common: prior studies indicate that 20%–30% of people referred to tertiary centres for presumed drug-resistant epilepsy go on to receive a PNES diagnosis.
Prolonged inpatient video-electroencephalography (vEEG), which records synchronised brain waves alongside behavioural data, is the diagnostic gold standard. However, it often costs over US$15,000 per admission in the United States, with reported waiting times of 8 to 12 weeks that vary across healthcare systems. Diagnostic delays can also lead to unnecessary treatment and postpone appropriate care.
Routine clinical notes contain extensive diagnostic signals, but seizure semiology (the symptoms, signs and sequence of a seizure-like event) overlaps considerably across disorders, and earlier narrative-based approaches have shown limited diagnostic performance.
How EpiScreen was developed
EpiScreen was trained and evaluated using 3,451 records from the Beth Israel Deaconess Medical Center’s Medical Information Mart for Intensive Care IV (MIMIC-IV) database and 10,182 records from the University of Minnesota (UMN) Clinical Data Repository. Labels reflected clinicians’ final documented diagnoses, and notes were processed to approximate the information available before diagnosis. People with coexisting epilepsy and PNES were excluded.
The system was built by instruction fine-tuning open-source LLMs, including Qwen-2.5-32B, DeepSeek-R1-32B, Llama-3.3-70B and Med42-70B, using Quantized Low-Rank Adaptation (QLoRA), a method that reduces the memory and training demands of adapting large models. To explain its reasoning, the researchers used sentence-level Integrated Gradients (IG), a technique that highlights the sentences most influential in the model’s decisions across 11 predefined clinical feature categories.
Four medical teams, made up of six neurology residents and two senior attending physicians, then evaluated 300 MIMIC-IV test cases, with each pair reaching decisions by consensus.
Study findings
EpiScreen consistently outperformed conventional supervised baselines, including ClinicalBERT. Its Med42-based variant achieved AUCs of 0.891 on MIMIC-IV and 0.976 on UMN, while the Qwen- and DeepSeek-based variants reached 0.992 on UMN. When tested on records from institutions that had not supplied their training data, the best AUCs reached 0.875 on MIMIC-IV and 0.980 on UMN.
Working alone, EpiScreen achieved 81.0% diagnostic accuracy, compared with 81.7% for unaided senior attending physicians and 76.3%–79.3% for unaided neurology residents. These differences were not statistically significant. With access to the AI’s predictions and sentence attributions, accuracy rose to 87.7%–89.0% across the resident groups and 89.3% for the attending physicians, with the largest improvement around 12.7 percentage points.
AI assistance also reduced false negatives (missed epilepsy cases) for resident Group A from 36 to 14, while the attending pair reduced false positives from 37 to 19.
Not all diagnostic labels were vEEG-confirmed, although sensitivity analyses in confirmed cases supported the findings. Documentation bias remains possible, and other seizure mimics, including syncope and sleep disorders, were not evaluated.
What this means for clinical practice
The authors suggest EpiScreen could be used during or after neurology referral, before confirmatory vEEG, to support earlier assessment and help clinicians prioritise urgent monitoring. They propose it could reduce diagnostic delays, costs and unnecessary exposure to antiseizure medication, though these remain potential benefits rather than measured outcomes.
The study highlights how AI tools that explain their reasoning may be most valuable when working alongside clinical judgement. Clinicians looking to build confidence in evaluating and using AI in practice may find The College of Contemporary Health’s AI Essentials for Primary Care short course a useful starting point.
Future research must prioritise prospective, real-time trials to confirm how EpiScreen performs in real-world settings using minimally curated notes. The tool is intended to support clinical judgement alongside definitive testing, not replace it.
CCH insight
As AI decision-support tools like EpiScreen move closer to everyday practice, healthcare professionals need the knowledge to assess what these systems can and cannot do. Our AI Essentials for Primary Care CPD course helps clinicians understand the role of AI in healthcare and apply it safely and confidently in practice.
Explore AI Essentials for Primary Care →
Source: npj Digital Medicine
Read More