Microsoft’s medical AI diagnoses 83% of simulated hard cases in Science study

Microsoft’s diagnostic AI correctly identified 83% of simulated complex medical cases in a study published in Science on October 22, compared with 36% for generalist doctors working individually. The result puts a system that coordinates several AI agents ahead of those physicians in this test—not ahead of the specialist teams that usually handle such difficult cases.

The cases came from the New England Journal of Medicine. The human comparison included primary care doctors and general internists, who could consult reference materials. Specialists and physician teams were not included. That is a material limit: cases this complex are usually handled by those specialists or teams, rather than a generalist working alone.

Called MAI-DxO, or DxO, Microsoft’s system organizes several AI agents into something resembling a panel of doctors. The agents question a patient, order tests and debate a diagnosis. The study therefore evaluated a coordinated diagnostic process, not simply a model answering a medical question in one response, according to TechCrunch’s account of the research.

How much did the AI panel add?

Microsoft tested the system with models from OpenAI, Google, Anthropic, Meta and DeepSeek. Its best results came with OpenAI’s GPT-5 and GPT-5.5. But the comparisons also separate two sources of performance: the underlying model’s ability and the extra benefit from Microsoft’s coordination system. TechCrunch noted that much of the improvement came from stronger underlying models, with DxO adding gains on top.

On complex cases, GPT-5 with a short, basic prompt already scored 78.9%. Adding DxO raised accuracy by about four percentage points. The larger change was in estimated cost per case: the standalone configuration was about $8,400, while DxO reduced that estimate by roughly $3,000. Microsoft assessed cost alongside accuracy, rather than treating a correct diagnosis as the only measure.

GPT-5.5 showed a different balance. On the complex cases, Microsoft reported that DxO lifted accuracy from 72% to nearly 87%, while slightly lowering estimated cost. Here, the coordination layer delivered a substantially larger accuracy gain than it did with GPT-5; the benefit was not uniform across the two models.

Complex-case accuracy, without and with DxO

GPT-5.5

72%→Nearly 87%

%

Microsoft reported a larger accuracy gain for GPT-5.5 on simulated complex cases, at slightly lower estimated cost.

The emergency-department cases produced a different result again. GPT-5.5 scored about 85% both with and without DxO, at similar cost. GPT-5 also saw little improvement in either accuracy or cost in that setting. The complex-case gains did not carry over into a comparable advantage on the emergency-department cases.

Microsoft’s researchers said accuracy gains naturally shrink as models approach the theoretical maximum. A company spokesperson also defended the system as a way to optimize accuracy and cost together. That is a narrower argument than saying the coordination layer always produces a more accurate diagnosis: some of its value can lie in the cost side of the comparison.

The next comparison moves into hospitals

Microsoft now plans to evaluate DxO’s clinical reasoning with Beth Israel Deaconess Medical Center in Boston. The planned evaluation will cover emergency and urgent-care settings. Dominic King, Microsoft AI’s vice president of Health, told TechCrunch that the technology was ready for real-world evaluation—a next step beyond the simulated cases in Science.

Patients will receive standard care, including triage, specialist attention, investigations, treatment and a discharge decision. Those who consent will also engage with DxO in parallel. Microsoft and its partners will compare the system’s recommendations with the doctors’ recommendations. The announced design keeps usual treatment in place; it is an evaluation of the system alongside clinical care, not an announcement that DxO will replace it.