BiologybioRxiv
Ranked by formulaVerdict: NotableDegree bias predicts, outperforms and improves machine learning phage-host interaction predictions
Machine learning (ML) models predicting phage-host interactions are widely used to prioritize phages for phage therapy.
Caveats
- Preprint; not yet peer reviewed.
VerdictWorth a reader's time today.
Abstract
Machine learning (ML) models predicting phage-host interactions are widely used to prioritize phages for phage therapy. The Area Under the Receiver Operating Characteristic (AUROC) metric is standard for model evaluation, with reported high scores (0.7 - 0.95) suggesting strong predictive power. However, because phage-bacterium networks are highly structured, these scores can be inflated by "degree bias", the baseline likelihood of infection driven purely by average phage infectivity and bacterial susceptibility. Here, we show that an interaction-naive model based solely on network degrees achieves higher global AUROC scores than five published ML models across ten datasets, indicating that global AUROC primarily reflects network structure rather than predicted interactions. Using randomized degree-conserved networks and degree-stratified evaluation, we isolate true interaction predictions and demonstrate that current ML model performance is far more modest than global AUROC suggests. Finally, we demonstrate that regressing out degree bias from training data improves interaction predictability and boosts global and stratified performance. Our work provides a critical framework for model evaluation and a straightforward route to improve ML-guided phage selection.
The editor's rubric
- Leverage3 / 5
- A method or resource many groups across the field will adopt within a year.
- Magnitude3 / 5
- Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.
- Evidence3 / 5
- Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
- Novelty3 / 5
- A genuinely new approach to an open problem.
- Trajectory2 / 5
- Some room to improve with obvious engineering.
- Stakes3 / 5
- Meaningful benefit to many people within a few years.
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we report); breadth (many tasks); gains (outperforms); novelty (alternative to status quo); design (randomized); verification (multiple benchmarks); stakes (global scale, major disease, prevention or cure).
How the score was computed
Desks place stronger verdicts first, then use this score to order work within the same verdict.
- Merit
- 5.8 / 10
- Adjusted merit
- 4.9 / 10
- Attention
- 0%
- Freshness
- 88%