Microsoft AI Beats Doctors in Simulated Diagnosis Test, Identifying 83 Percent of Complex Cases

Microsoft's new artificial intelligence system has far outperformed generalist doctors in a study of complex medical diagnosis. In research published on October 22 in the journal Science, the system correctly identified 83 percent of complex medical cases, compared with 36 percent accuracy for generalist physicians who could consult reference materials.

Microsoft calls the technology the MAI Diagnostic Orchestrator, or MAI-DxO. The system uses multiple AI agents that work together much like a panel of doctors discussing a difficult case. It begins with limited information, asks follow-up questions, considers which medical tests might help, and its agents challenge one another's conclusions as new information becomes available.

Researchers evaluated the technology using 304 complex medical cases from The New England Journal of Medicine. These cases often involve unusual illnesses that challenge even experienced physicians, and MAI-DxO correctly diagnosed approximately 83 percent of them.

Without reference resources, the generalist physicians' accuracy dropped to 22 percent. Researchers also tested the system using historical emergency department cases, where the AI achieved approximately 80 percent accuracy, compared with 47 percent for the physician comparison group.

There are important limits behind those numbers. The physicians in the comparison were generalists, including primary care doctors and general internists, who evaluated cases individually. Hospitals typically involve specialists when patients develop particularly difficult problems, and Microsoft did not directly compare its system against teams of medical specialists.

There is another important detail: Microsoft's AI was evaluated across 304 complex cases, while physicians worked through a smaller set of 56 cases held out from the system's development process, so the headline percentages do not represent two groups completing exactly the same number of cases. The research also relied on existing medical cases rather than treating patients in real time.

The study's authors also work for Microsoft, making independent testing especially important before hospitals consider relying on the technology for patient care.

The underlying AI models played a major role as well. OpenAI's GPT-5 correctly solved approximately 79 percent of the complex cases with a basic prompt without Microsoft's orchestration system, and adding MAI-DxO increased accuracy to approximately 83 percent. With the newer GPT-5.5 model, the orchestration system raised accuracy from approximately 72 percent to nearly 87 percent.

On cost, researchers assigned estimated prices to diagnostic procedures. In one cost-conscious configuration, MAI-DxO achieved approximately 81 percent accuracy on complex cases while generating an estimated $2,241 in diagnostic costs per case, against an average of approximately $4,725 for the physicians using reference materials. Some configurations designed to maximize accuracy generated higher testing costs.

Dr. Marc Siegel, Fox News Senior Medical Analyst, advocates for AI as a tool to enhance, not replace, human doctors, stressing the irreplaceable value of personal interaction and physician oversight for diagnoses.

Microsoft CEO of AI Mustafa Suleyman has described the company's larger ambition as “medical superintelligence.” Microsoft is also collaborating with Mayo Clinic on AI technology that could eventually help clinicians anticipate how a patient's medical condition might develop.