BiologybioRxiv
Ranked by formulaVerdict: NotableRepresentation-aware calibration enables statistically reliable protein structure search
Searching large databases of predicted protein structures can reveal remote homologs, but reliable discovery requires accurate significance estimates.
Key numbers
- 18.2% with slightly fewer false
- 23.5% at the same cutoff
Caveats
- Preprint; not yet peer reviewed.
VerdictWorth a reader's time today.
Abstract
Searching large databases of predicted protein structures can reveal remote homologs, but reliable discovery requires accurate significance estimates. We show that native E-values reported by protein structure search tools such as Foldseek and Reseek can be unreliable, leading to false discoveries or missed homologs. To address this problem, we introduce Struct-Caliper, which provides reliable query-specific p-values for protein structure search. The method uses representation-aware query randomization to generate background queries and compares observed alignment scores with the highest scores from randomized searches of the same database. In simulations, SG p-values generally kept reported false positive at or below the chosen cutoff under the evaluated randomization procedures. In searches of 15,177 ASTRAL40 structures, SG p-values increased Foldseek homolog recovery by 18.2% with slightly fewer false discoveries compared with native E-values at a cutoff of 0.01. For Reseek, false discoveries even decreased by 99.5%. Across 73 queries searched against more than two million AlphaFoldDB structures, SG p-values increased the mean recovery of DALI-supported matches by 23.5% at the same cutoff. These results show that Struct-Caliper can improve the reliability of structure searches and help prioritize discoveries for further biological analysis.
The editor's rubric
- Leverage3 / 5
- A method or resource many groups across the field will adopt within a year.
- Magnitude3 / 5
- Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.
- Evidence3 / 5
- Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
- Novelty3 / 5
- A genuinely new approach to an open problem.
- Trajectory2 / 5
- Some room to improve with obvious engineering.
- Stakes2 / 5
- Benefits a professional community (practitioners, clinicians, engineers).
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose, we report); gains (relative gain); novelty (discovery); design (randomized); stakes (global scale).
How the score was computed
Desks place stronger verdicts first, then use this score to order work within the same verdict.
- Merit
- 5.6 / 10
- Adjusted merit
- 4.7 / 10
- Attention
- 0%
- Freshness
- 88%