Microsoft says its new AI system diagnoses patients four times more accurately than human doctors
Microsoft has taken it “A real step towards medical superintelligence,” said Mustafa Suleyman, CEO of the company’s artificial intelligence division. The tech giant said its powerful new AI tools could diagnose four times more accurately than a human physician team, and cost much less.
The experiment tested whether the tool could correctly diagnose patients with the disease, mimicking what human doctors usually do.
The Microsoft team used 304 case studies from the New England Journal of Medicine to design a test called the Sequential Diagnostic Benchmark (SDBENCH). The language model divides each case into a step-by-step process, which the doctor will perform for diagnosis.
Then, Microsoft researchers built a system called MAI Diagnostic Orchestration (MAI-DXO), which asked about a variety of leading AI models, including Openai’s GPT, Google’s Gemini, Anthropic’s Claude’s Claude, Meta’s Llama and Xai’s Grok, a way to cutely mimic several human experts.
In their experiments, Mai-Dxo performed better than human doctors, with an accuracy of 80% compared to 20% of doctors. It also reduces the cost by 20% by choosing cheaper tests and procedures.
“This orchestration mechanism – multiple agents working together in this continuous style – will bring us closer to medical super-intelligence,” Suleiman said.
The company poached several Google AI researchers to help the effort — but another sign of a war on advanced AI expertise in the technology industry. Suleyman was a former Google executive working in AI.
AI has been widely used in certain parts of the U.S. healthcare industry, including helping radiologists explain scans. The latest multimodal AI models have the potential to act as a more general diagnostic tool, although the use of AI in healthcare can raise your own problems, especially with bias in training data towards demographic specificity.
Microsoft has not yet decided whether it will try to commercialize the technology, but the same executive speaking anonymously said the company could integrate it into Bing to help users diagnose the disease. The company can also develop tools to help medical experts improve or even automate patient care. “What you will see over the next few years is that more and more of our work is proving these systems in the real world,” Suleiman said.
The project is the latest in a growing number of research that demonstrates how AI models diagnose diseases. Over the past few years, both Microsoft and Google have published papers showing that large language models can accurately diagnose disease when medical records are obtained.
The new Microsoft research is different from previous work because it can be analyzed by analyzing symptoms, ordering tests and performing further analysis until the diagnosis is reached. Microsoft describes in a blog post about the project how it combines several cutting-edge AI models into a “path of super-intelligence in healthcare.”
The project also shows that AI can help reduce health care costs, which is a key issue, especially in the United States. “Our model performs well, both for diagnosis and very effective diagnosis,” said Dominic King, Microsoft vice president who works on the project.
