Sequential diagnosis with language models

We explored the capabilities of frontier language models in sequential diagnosis, a diagnostic paradigm that captures the iterative nature of clinical reasoning.

By Nori, Daswani, Kelly +12Science (New York, N.Y.)Oct 22, 2026

Score█████░░░░░4.9

Key numbers

83.2% accuracy on New England
80.3% on MIMIC-IV-ED emergency department

VerdictWorth a reader's time today.

Read the original

Abstract

We explored the capabilities of frontier language models in sequential diagnosis, a diagnostic paradigm that captures the iterative nature of clinical reasoning. Given a brief clinical presentation, clinicians selectively acquired information to reach a diagnosis, enabling assessment of diagnostic accuracy and workup cost. We developed an interactive benchmark and an artificial intelligence (AI) system that simulates a panel of physicians, continually revising the differential diagnosis and selecting cost-effective tests. Using this benchmark, the system achieved 83.2% accuracy on New England Journal of Medicine clinicopathological conference cases and 80.3% on MIMIC-IV-ED emergency department admissions, outperforming generalist physicians (36.1 and 46.8%, respectively). Although these findings were obtained in a controlled benchmark setting, they highlight the promise of language models for supporting more accurate, lower-cost diagnoses.

Harsha Nori, Mayank Daswani, Christopher Kelly, Scott Lundberg, Marco Tulio Ribeiro, Marc Wilson, Xiaoxuan Liu, Viknesh Sounderajah, Jonathan M Carlson, Matthew P Lungren, Bay Gross, Peter Hames, Mustafa Suleyman, Dominic King, Eric Horvitz

The editor's rubric

Heuristic review

Leverage2 / 5: Reusable within one subfield (a technique, dataset, or protocol a few groups will adopt).; 10% of this desk’s rubric
Magnitude3 / 5: Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.; 20% of this desk’s rubric
Evidence3 / 5: Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.; 32% of this desk’s rubric
Novelty3 / 5: A genuinely new approach to an open problem.; 8% of this desk’s rubric
Trajectory2 / 5: Some room to improve with obvious engineering.; 5% of this desk’s rubric
Stakes2 / 5: Benefits a professional community (practitioners, clinicians, engineers).; 25% of this desk’s rubric

Dimension	Level	Weight	What that level means
Leverage	██░░░2/5	10%	Reusable within one subfield (a technique, dataset, or protocol a few groups will adopt).
Magnitude	███░░3/5	20%	Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.
Evidence	███░░3/5	32%	Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
Novelty	███░░3/5	8%	A genuinely new approach to an open problem.
Trajectory	██░░░2/5	5%	Some room to improve with obvious engineering.
Stakes	██░░░2/5	25%	Benefits a professional community (practitioners, clinicians, engineers).

Editor’s rationale

Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: gains (outperforms).

How the score was computed

rank-2026-10-18

Score█████░░░░░4.9

Score = 10 × (80% × adjusted merit / 10 + 10% × attention + 10% × freshness)

Desks place stronger verdicts first, then use this score to order work within the same verdict.

Merit: 5.2 / 10; Weighted rubric, evidence-gated.
Adjusted merit: 4.4 / 10; Shrunk toward the desk prior by editor confidence (32%).
Attention: 59%; Citations, upvotes, points, mentions.
Freshness: 80%; Half-life decay since publication.

Citations27 (reference 20, via openalex, Oct 25, 2026, 06:17 UTC)

The record

Reviewed by heuristic-v10 on Oct 25, 2026, 06:17 UTC. Paper type: empirical.
Categories: Journal Article, Humans, Diagnosis, Differential, Artificial Intelligence, Benchmarking, Large Language Models
BRIEF, No.5 in the Medicine edition of October 25, 2026.