AI Recommends Care Costing Three Times More Than Doctors, Study Finds

Key Takeaways:

• A new study published in NEJM AI found that care recommended by large language models (LLMs) cost around 3.2 times more than the care actually recommended by treating physicians – US$314 per visit compared with US$97.
  
• The extra cost was split almost evenly between additional diagnostic tests and specialist referrals, and in every visit where physicians ordered no tests, every AI system suggested testing anyway.
  
• Prompting the models to be “cost-aware” or “parsimonious” narrowed the gap but did not close it, with researchers concluding that LLM outputs should remain suggestions and that final decisions should stay with clinicians.
  
  
  

Why this study matters now

AI alignment – the question of whether AI systems act in line with the values and priorities of the people and organisations using them – has become one of the most debated topics in healthcare technology. A timely peer-reviewed study in NEJM AI has now added some much-needed empirical evidence to that debate by comparing the cost of care recommended by LLMs with the care actually provided by treating physicians.

The headline finding is striking: LLM-recommended care was about 3.2 times as expensive as physician care.

While LLMs are increasingly being consulted for clinical decision support, the researchers noted that the cost of the services and treatments these systems recommend had not previously been systematically evaluated.

How the study was carried out

Researchers evaluated 24 AI systems, made up of 20 general-purpose LLMs and four medical-specific systems. These were tested on 200 deidentified outpatient and emergency department notes from MTSamples, a public collection of deidentified US-format clinical notes contributed by practising clinicians.

Each AI system was given the clinical presentation as it was recorded before the physician’s assessment, and was then asked to write its own plan. The physician’s documented plan served as the comparator.

Orders from both the AI systems and the physicians were extracted by three independent LLMs and priced at 2026 Medicare rates. The primary outcome was the total per-visit cost of recommended care, defined as diagnostic tests plus specialist referrals. New medications were counted but not priced.

The researchers also tested two alternative prompts: one cost-aware prompt and one “parsimonious-but-safe” prompt.

AI-recommended care cost more than three times as much

Across the 20 general-purpose systems, AI-suggested care cost a mean of US$314 per visit, compared with US$97 for care recommended by treating physicians. This represented a difference of US$216 per visit (95% confidence interval, US$177 to US$257; P<0.001), or 3.2 times the physicians’ cost.

Mean costs varied considerably between AI systems, ranging from US$209 to US$599 per visit.

The difference was driven by two main factors in roughly equal measure:

• Diagnostic tests: US$105 of the extra cost per visit
• Specialist referrals: US$111 of the extra cost per visit

The AI systems also recommended more new medications than physicians, at 1.14 per visit compared with 0.59.

Importantly, the four specialised medical systems performed no better on cost than their general-purpose counterparts, recommending care that was between 2.7 and 3.5 times as expensive as physician care.

LLMs struggle to recommend doing nothing

One of the clearest differences between the AI systems and physicians was their approach to visits where no testing may have been needed. Physicians recorded no diagnostic testing in 66% of visits – around two-thirds of encounters. On those same visits, every AI system suggested some form of testing.

These were usually stereotyped panels of standard tests, such as metabolic panels, haemoglobin A1c and lipids.

The additional tests were judged to be guideline-concordant in only about 20% of the cases where the AI ordered more than the physician. The remainder represented additional testing that was not supported by guidelines.

Better prompting helped – but did not solve the problem

The researchers found that the way the AI systems were prompted made a meaningful difference to the cost of the care they recommended, although it did not remove the excess entirely:

• Telling the models to be “cost-aware” reduced the gap to 2.5 times the physicians’ cost.
• A “parsimonious” prompt trimmed the gap further, to 1.4 times the physicians’ cost.

In their conclusions, the study authors stated that the care these systems recommended cost substantially more than physicians’ documented care, chiefly because they suggested testing at visits where the physician had ordered none. They added: “LLM outputs are suggestions, not plans of care; final decisions should remain with clinicians.”

Dr Robert Wachter: encoding “what is enough”

The study arrived shortly after Dr Wachter, Chair of Medicine at the University of California, San Francisco (UCSF), published an essay on the AI alignment problem in healthcare. He noted that the new research provided “some empirical meat to put on its bones.”

Dr Wachter agrees with the study authors on several key points:

• The findings do not argue against clinical AI; rather, its value depends on how it is deployed.
• The next advance should be systems that recognise “what is enough while balancing patient safety with parsimony.”

However, he differs on how that goal should be reached. In Dr Wachter’s view, “what is enough” is not something AI will discover on its own. Instead, it is a value judgement that health systems, payers, regulators and AI developers “will have to encode deliberately.”

What this means for clinicians

For healthcare systems already facing very high levels of spending, AI recommending care that is around three times more expensive than that recommended by physicians is unlikely to be welcome news. That said, practising differently is not necessarily the same as practising worse, and the study leaves open the question of whether AI is tilting towards wastefulness or towards thoroughness.

What the research does make clear is that clinicians will need to apply their own judgement when reviewing AI-generated suggestions, rather than accepting them at face value. For primary care professionals who want to build their confidence in understanding and critically appraising AI tools in practice, The College of Contemporary Health’s AI Essentials for Primary Care course offers a practical starting point.

CCH insight

As AI tools become a routine part of clinical decision support, knowing when to question a recommendation is becoming an essential skill. Our AI Essentials for Primary Care CPD course helps healthcare professionals understand how AI works in a primary care setting, so they can use it safely and effectively while keeping clinical judgement at the centre of person-centred care.

Explore AI Essentials for Primary Care →

Source: NEJM AI