Microsoft claims new AI tool can diagnose complex cases better than doctors

 Key Takeaways:

• Microsoft’s new AI “Diagnostic Orchestrator” outperformed doctors by a factor of four when diagnosing intricate medical cases, achieving 85.5% accuracy versus around 20% for clinicians under restricted conditions.
• The system employs an innovative method where multiple AI agents debate and collaborate, offering a transparent “chain of debate” reasoning process.
• Though promising in simulated trials, experts caution the technology remains at an early stage, untested in real-world clinical practice and not yet peer reviewed.
  

Microsoft unveils AI tool it says could revolutionise medical diagnosis

Microsoft has revealed an artificial intelligence–driven medical tool that it claims is four times more successful than human doctors at diagnosing complex health conditions. The development, which the company hopes will accelerate treatment pathways, was announced alongside research it believes could signal a major step towards so-called “medical superintelligence”.

The new system, named the Microsoft AI Diagnostic Orchestrator (MAI-DxO), represents the first major output from Microsoft’s dedicated AI health unit. This division was established last year under the leadership of Mustafa Suleyman, who co-founded DeepMind, the AI research lab now owned by Google. Microsoft attracted staff from DeepMind to join this effort.

Towards medical superintelligence

In an interview with the Financial Times, Mustafa Suleyman, now chief executive of Microsoft AI, described the results as an early milestone on the road to vastly more capable AI systems that could help ease staffing shortages and cut waiting times across pressured healthcare systems.

Suleyman said:

“Microsoft is nearing AI models that are not just a little bit better, but dramatically better, than human performance: faster, cheaper and four times more accurate. That is going to be truly transformative.”

How the “orchestrator” works

The technology’s backbone is an AI “orchestrator” that assembles virtual panels comprising five distinct AI agents, each acting like a specialised doctor. These agents take on roles such as formulating diagnostic hypotheses or selecting appropriate tests. They then interact and effectively “debate” among themselves before reaching a collective diagnostic decision.

This innovative framework employs a method called “chain of debate”, which compels the AI models to articulate step by step how they arrive at conclusions, offering unprecedented transparency into machine reasoning.

Testing against medical cases

To assess the system’s abilities, Microsoft researchers used 304 documented case studies drawn from the New England Journal of Medicine (NEJM). These cases represent some of the most challenging diagnostic puzzles solved by doctors in practice.

The orchestrator integrated leading large language models (LLMs) from OpenAI, Meta, Anthropic, Google, xAI and DeepSeek. Although the orchestrator improved the performance of all these models, it achieved the highest success rate when paired with OpenAI’s “o3” reasoning model – correctly diagnosing 85.5% of the NEJM cases.

For comparison, experienced physicians – who in this test were not allowed to consult textbooks or colleagues, factors that would likely have raised their success rates – solved around 20% of these same cases.

Potential integration and cost efficiencies

Microsoft indicated that elements of this diagnostic technology could soon be incorporated into its Copilot AI chatbot and Bing search engine, which currently process approximately 50 million health-related queries daily.

Dominic King, formerly the head of DeepMind’s health team and who joined Microsoft late last year, noted:

“The programme had performed better than anything we’ve ever seen before. There is an opportunity here today to act almost as a new front door to healthcare.”

King also highlighted that the AI agents were prompted to consider costs, which led to fewer diagnostic tests being ordered. In some scenarios, this resulted in hundreds of thousands of dollars in theoretical savings during the trial simulations.

Caution from experts and early-stage caveats

However, King was careful to stress that the technology remains in its infancy. The work has not yet undergone peer review and is not ready for deployment in live clinical environments.

Eric Topol, a cardiologist and director of the Scripps Research Translational Institute, commented:

“This is a landmark study. While this work was not done in the setting of real world medical practice, it is the first to provide evidence for the efficiency potential of generative AI in medicine – accuracy and cost savings.”

Microsoft’s broader AI strategy and industry dynamics

This effort also arrives as Microsoft continues to deepen its investment in AI. The company has put nearly $14 billion into OpenAI and holds exclusive rights to use and sell its technology. Nonetheless, the partnership faces tensions as OpenAI seeks to shift towards a for-profit model, with negotiations ongoing about future terms.

Suleyman remarked that although OpenAI’s model delivered the strongest performance, Microsoft remains flexible regarding which of the major models underpin the orchestrator:

“We have long believed that they’ll become commodities … it’s the aggregate orchestrator which I think is the differentiator.”

A sign of things to come?

The launch of MAI-DxO follows in the wake of other AI–healthcare breakthroughs, notably from Suleyman’s former organisation, DeepMind. Last year, the lab’s chief Sir Demis Hassabis was jointly awarded a Nobel Prize in chemistry for work using AI to unravel the structures of proteins fundamental to life.

For now, Microsoft’s technology stands as a striking example of AI’s growing capacity to tackle some of healthcare’s most demanding challenges. Yet its real-world impact will depend on rigorous validation and careful integration into the delicate fabric of clinical care.