Microsoft's AI Panel Beats Generalists on Complex Diagnoses

An AI diagnostic system developed by Microsoft correctly identified complex medical conditions in a simulation 83% of the time, a figure that is more than double the 36% success rate achieved by primary care doctors and general internists. The study, published in Science on Thursday, highlights a significant gap between algorithmic reasoning and individual human judgment in difficult clinical scenarios. Mustafa Suleyman, CEO of Microsoft AI, framed these results as a critical step toward achieving what he calls "medical superintelligence."

The system, known as MAI-DxO, functions by coordinating multiple AI agents that mimic a panel of physicians. These agents engage in questioning, ordering tests, and debating potential diagnoses. While the accuracy figures are striking, the study’s design compares the AI against individual generalists working with reference materials, rather than against the specialist teams or physician groups who typically handle such complex cases in real-world settings. Dominic King, Vice President of Health at Microsoft AI, noted that the growing mismatch between medical expertise supply and demand is the primary challenge the technology aims to address.

Under the hood, the performance gains appear driven largely by the power of the underlying models rather than the orchestration layer alone. Microsoft tested the system with models from OpenAI, Google, Anthropic, Meta, and DeepSeek, finding that the highest results were achieved with OpenAI’s GPT-5 and GPT-5.5. For instance, when using GPT-5.5 on complex cases, the DxO orchestration boosted accuracy from 72% to nearly 87%. However, on emergency department cases, the addition of the orchestration layer provided little to no improvement in either cost or accuracy compared to using the base model alone.

The cost-efficiency of the system also varies depending on the complexity of the case and the base model used. With GPT-5, the orchestration reduced the estimated cost per case by approximately $3,000 while raising accuracy by only about four points. When using the less capable Gemini 2.5 Flash model, the orchestration helped narrow the gap between weaker and stronger models, though the baseline difference remained larger than the boost provided by the system. Researchers indicated that as models approach their theoretical performance limits, the marginal gains from orchestration naturally diminish.

Looking ahead, Microsoft is partnering with the Mayo Clinic to train a dedicated healthcare foundation model, with a release planned for next year. Suleyman predicts that by 2027, the company will be on the verge of a "coding style breakthrough" for medical records, allowing for real-time predictions of patient outcomes. This initiative coincides with a new partnership with Beth Israel Deaconess Medical Center, where the system will undergo real-world evaluation across emergency and urgent care settings. King described the upcoming study as the largest of its kind, aiming to provide compelling evidence for the broader deployment of these diagnostic tools.

Microsoft’s efforts place it in direct competition with Google and OpenAI, both of which have recently claimed that their AI systems can match or outperform human doctors. By defining "medical superintelligence" as the capacity to surpass human-led health systems in reasoning and coordination, Microsoft aims to move beyond simple benchmarking and integrate deeply into clinical workflows. Suleyman, who joined Microsoft to lead its AI unit in 2024, believes that healthcare models will improve by an order of magnitude in capability within the next 24 to 36 months, driven by real-world usage data and iterative refinement.