83.2 percent versus 22.3 percent. That’s the number making the rounds in tech and health media right now. A Microsoft AI system scored dramatically higher than unaided primary care doctors on complex medical case studies. The system is called MAI-DxO, and it’s one of the most striking demonstrations yet of AI in medicine. But the numbers deserve a closer look before anyone calls this a medical revolution.
Microsoft AI built MAI-DxO (the MAI Diagnostic Orchestrator) to coordinate several AI agents that work together like a virtual panel of physicians. The agents ask questions, order tests, and debate a diagnosis before committing to an answer. The underlying study was published on October 22, 2026 in the prestigious journal Science, and it has since sparked debate about what role artificial intelligence could play in clinical diagnostics going forward.
Table of Contents
Key Takeaways
- MAI-DxO hit 83.2 percent diagnostic accuracy across 304 complex NEJM case studies, versus just 22.3 percent for unaided primary care doctors.
- The study was published October 22, 2026 in Science; all 15 authors are Microsoft employees, including Microsoft AI CEO Mustafa Suleyman.
- On emergency department cases from the MIMIC-IV-ED dataset, MAI-DxO reached up to 80.3 percent versus 46.8 percent for the participating physicians.
- The comparison group consisted of generalist physicians, not the specialists who would normally handle cases this complex in real practice.
- Clinical trials are set to begin within the next two months at Beth Israel Deaconess Medical Center.
What Exactly Is MAI-DxO?
MAI-DxO doesn’t work like a single chatbot model answering one question. Instead, the system orchestrates multiple language-model agents that act together like an interdisciplinary medical team. One agent might formulate history-taking questions, another proposes lab tests, a third checks the plausibility of the emerging diagnosis. This interplay is meant to offset the weaknesses of any single model and lead to more robust decisions.
The paper, “Sequential diagnosis with language models,” appeared in Science, Volume 394, Issue 6822, pages 423-428. Among its 15 authors are Mustafa Suleyman alongside longtime Microsoft researchers Dominic King and Eric Horvitz, both with a focus on AI and healthcare.
“A step toward medical superintelligence” is how Mustafa Suleyman described the study’s result.
That phrasing is deliberately ambitious. It signals Microsoft’s intent to build not just another diagnostic tool, but systems that could eventually exceed human expertise in medicine by a wide margin.
The Study Results in Detail
Researchers tested MAI-DxO against 304 complex case write-ups from the New England Journal of Medicine, one of the most respected medical journals in the world. These cases are considered especially demanding, often describing rare or ambiguous symptom constellations.
Results on NEJM cases:
| Group | Accuracy |
|---|---|
| MAI-DxO | 83.2 percent |
| Physicians with reference material | 36.1 percent |
| Physicians without aids | 22.3 percent |
| Best individual physician | 51.8 percent |
Emergency department cases from the MIMIC-IV-ED dataset told a similar story. MAI-DxO reached 80.3 percent accuracy without budget constraints and 73.2 percent under a cost constraint. The participating physicians averaged 46.8 percent, with the best individual doctor reaching 62.5 percent.
These numbers are impressive, but they describe a simulated environment, not a real clinical encounter with an actual patient, a physical exam, and all the uncertainty of everyday practice.
Which Base Model Performs Best?
Microsoft’s researchers tested MAI-DxO across several AI base models, including systems from Anthropic, Meta, and DeepSeek. The best results came from pairing the orchestrator with OpenAI’s GPT-5 and GPT-5.5.
The gap between a model running alone and the same model with orchestration is revealing:
- GPT-5 alone reached 78.9 percent at an estimated $8,400 per case. With MAI-DxO, cost dropped by roughly $3,000 while accuracy rose by about 4 percentage points.
- GPT-5.5 alone scored 72 percent. With DxO orchestration, that climbed to nearly 87 percent, at a slightly lower cost.
- Google Gemini 2.5 Flash, a noticeably weaker model, improved from 46.4 percent alone to almost 69 percent with DxO.
The pattern here matters: the weaker the underlying model, the bigger the relative benefit from orchestration. For top-tier models like GPT-5, the gain from MAI-DxO is comparatively modest, while weaker models like Gemini 2.5 Flash see a far more pronounced improvement.
A Critical Look at the MAI-DxO Study
As impressive as these numbers look at first glance, some sober context is needed.
First, this is a simulation study. No real clinical application with actual patients was tested — researchers analyzed written case descriptions. The complexity of a real doctor’s visit, with a physical exam, body language, tone of voice, and spontaneous follow-up questions, can only be partially captured this way.
Second, the comparison group matters a great deal. The tested cases come from the NEJM and would normally be handled in real clinical practice by specialists or interdisciplinary physician teams, not by individual generalists. Comparing a highly sophisticated AI system against a single primary care doctor without specialist training isn’t the same as asking whether AI outperforms the medical profession as a whole.
Third, the benefit of orchestration shrinks for stronger models. While MAI-DxO boosted accuracy by more than 20 percentage points for a weaker model like Gemini 2.5 Flash, the gain for GPT-5 was only about 4 percentage points. That suggests a large share of the performance actually comes from the underlying language model itself, not exclusively from the orchestration architecture.
None of this diminishes the technical achievement of MAI-DxO, but it does complicate the “AI beats doctors” headline into a more nuanced picture.
Background: From Early Prototype to MAI-DxO
MAI-DxO isn’t a sudden breakthrough — it’s the product of several years of research. Microsoft first introduced an earlier version of the system in mid-2025. That early variant, paired with OpenAI’s o3 model, already reached up to 85.5 percent accuracy on comparable test cases.
The current Science publication builds on that earlier work, adding more systematic tests across multiple base models and incorporating emergency department data from MIMIC-IV-ED. That gives the study a substantially broader foundation than the original 2025 presentation.
What’s Next: Clinical Trials at Beth Israel Deaconess Medical Center
Perhaps the clearest signal of real-world relevance is the announcement that clinical trials are set to begin within the next two months at Beth Israel Deaconess Medical Center. The Boston hospital is affiliated with Harvard Medical School and is an established research institution within the US healthcare system.
This marks the transition from a pure simulation study to actual clinical testing. Only in that context will it become clear whether the impressive lab numbers hold up under real-world conditions — with actual patients, incomplete information, and time pressure.
What Does This Mean in Practice?
More on AI agents: On the topic of autonomous AI systems and their risks, it’s also worth reading about how OpenAI’s own AI agents disabled their safety monitoring.
For patients and medical professionals in Germany and elsewhere outside the US, MAI-DxO is not yet an available tool. It remains a research system, primarily tested in the United States. Still, the development is worth watching for several reasons:
- AI-assisted diagnostic support could find its way into clinical decision-support systems in the medium term.
- Orchestrating multiple AI agents as a method could prove useful well beyond medicine.
- The debate over AI’s role in safety-critical domains like medicine is only intensifying because of studies like this one.
What’s important: MAI-DxO does not replace a medical examination and should not be mistaken for a substitute for professional medical advice. The study itself emphasizes the experimental nature of the results.
Bottom Line
The MAI-DxO study delivers striking numbers: 83.2 percent accuracy on complex NEJM cases versus 22.3 percent for unaided primary care doctors is a substantial gap. At the same time, the critical analysis shows that the comparison group consisted of generalists, while the tested cases would normally be handled by specialists in real-world practice. And the benefit of orchestration shrinks considerably for the strongest models, like GPT-5, compared to weaker ones.
The upcoming clinical trials at Beth Israel Deaconess Medical Center will show whether the lab results hold up in real patient care. Until then: MAI-DxO is a promising research development, but not yet a production-ready medical tool. Anyone following the future of AI-assisted medicine should keep an eye on the clinical trial results in the coming months.
