AI Matches Doctors on Tough Cases

Microsoft says a new system developed by its artificial intelligence division can diagnose difficult medical cases with significantly more accuracy than general practitioners. The company published research in the journal Science on Thursday stating that its tool, called MAI-DxO, was correct in 83% of complex cases during a simulation. By comparison, primary care doctors and general internists working individually with reference materials were correct 36% of the time.

However, the study notes that cases of this difficulty are typically referred to specialists or physician teams, who were not part of the comparison. The MAI-DxO system operates by coordinating multiple AI agents that function like a panel of doctors, questioning a patient, ordering tests, and debating a potential diagnosis.

Mustafa Suleyman, who leads Microsoft's AI unit, characterized the results as a step toward building medical superintelligence. He stated that the company is collaborating with the Mayo Clinic to train a foundation model for healthcare, which is scheduled for release next year. Suleyman expects significant progress in medical records by 2027, predicting the ability to stream highly accurate forecasts regarding the unfolding of a medical record.

The company acknowledged that accuracy improvements naturally decrease as AI models approach maximum performance levels. While the tool assists with optimizing both accuracy and cost, Microsoft stated that it is focused on integrating with clinicians and healthcare systems rather than simply answering medical exam questions. Suleyman said the company aims to offer the absolute number one state-of-the-art medical clinical decision support products over the next few years.

To test the system in a real-world setting, Microsoft will evaluate DxO's clinical reasoning capabilities through a partnership with Beth Israel Deaconess Medical Center in Boston. Patients receiving standard treatment, including triage and time with a specialist, will be compared against DxO's recommendations to assess the technology's ability to be deployed at scale.