BiologybioRxiv
Ranked by formulaVerdict: NotableLanguage models annotate mass cytometry clusters within the range where human experts disagree
Cell-type identification of clusters from unsupervised cytometry pipelines (e.g.
Key numbers
- 98% metacluster-level concordance
- 99% of major-lineage assignments but
- 73% of T-cell maturation states
Caveats
- Not randomized: its findings are associations.
- Preprint; not yet peer reviewed.
VerdictWorth a reader's time today.
Abstract
Cell-type identification of clusters from unsupervised cytometry pipelines (e.g. FlowSOM, GigaSOM) is a time-consuming, expert-dependent bottleneck, typically requiring an expert to read a heatmap of median marker intensities and assign a label to each cluster. Large language models (LLMs) can annotate single-cell RNA-seq clusters from marker-gene lists, but cytometry cluster data differ in both dimensionality and data type. We present a direct procedure in which cluster medians are scaled per marker and grouped by similarity into a few dozen metaclusters. The continuous marker profiles are then passed to the LLM, which returns a cell-type label and confidence level for each metacluster in a single call, without a gating hierarchy, manually defined marker-specific thresholds, or task-specific training data. Across four human and mouse datasets spanning blood, bone marrow and gut, a single well-chosen model reproduces the reference annotation at the major-lineage level with 85-98% metacluster-level concordance (cloud Claude Sonnet 4.6 85-98%, best local model 83-96%), substantially exceeding majority-class baselines; chance-corrected agreement is Cohen's k = 0.67-0.83. Local open-weight models perform competitively with the cloud model on the independent benchmarks, enabling application to clinical or patient-derived data subject to data-protection restrictions. Reliability is model-dependent: no local model is best everywhere, and the frontier model Claude Opus 5 was no more reliable than Claude Sonnet 4.6, the most consistent in concordance across datasets. Because no annotation is absolute ground truth, we obtained two additional expert annotations of the same continuous heatmaps, blinded to the reference and to all model output, to establish a method-matched agreement ceiling. The experts agreed on ~99% of major-lineage assignments but only ~73% of T-cell maturation states; on that axis the best models fall within the expert disagreement range. LLM-based annotation is therefore a practical, training-free alternative to rule-based annotation, provided its output is checked against the intended panel and species.
The editor's rubric
- Leverage3 / 5
- A method or resource many groups across the field will adopt within a year.
- Magnitude3 / 5
- Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.
- Evidence3 / 5
- Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
- Novelty2 / 5
- A new combination of known ideas.
- Trajectory2 / 5
- Some room to improve with obvious engineering.
- Stakes2 / 5
- Benefits a professional community (practitioners, clinicians, engineers).
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); gains (outperforms); design (blinded); scale (scalable). Red flags: not randomized.
How the score was computed
Desks place stronger verdicts first, then use this score to order work within the same verdict.
- Merit
- 5.2 / 10
- Adjusted merit
- 4.5 / 10
- Attention
- 0%
- Freshness
- 88%