BiologybioRxiv
Ranked by formulaVerdict: NotableCan Folding Models Tell Binders from Bluffers? Evidence from POISK: The Patent-Derived Antibody Dataset
Structure prediction models are increasingly deployed as zero-shot digital screens in antibody design, operating under the assumption that high folding confidence implies true binding.
Key numbers
- 72% pairwise accuracy over all
- 87% on the most confident
- 29% of trials
Caveats
- Preprint; not yet peer reviewed.
VerdictWorth a reader's time today.
Abstract
Structure prediction models are increasingly deployed as zero-shot digital screens in antibody design, operating under the assumption that high folding confidence implies true binding. However, rigorous validation of this premise has been limited by small test sets, narrow target diversity, and the risk of data leakage from training-adjacent structures. Here we introduce POISK: Patent-extracted Organization of Immunoglobulin Sequence Knowledge, a large-scale dataset derived from the US patent literature, comprising 89074 antibody-antigen records extracted from 10110 patents using an LLM-based pipeline. From this resource, we curate a balanced benchmark set of 22494 complexes: 11247 patent-validated binders without a known resolved structure paired with putative non-binders constructed from sequence-similar human repertoire antibodies. Using this dataset, we benchmark six leading structure prediction models (AlphaFold-Multimer, Boltz-2, IntelliFold v2, OpenFold3, Protenix-v1, and Protenix-v2) on their ability to discriminate binders from decoys. The best-performing model, Protenix-v2, achieves 72% pairwise accuracy over all pairs and 87% on the most confident half of predictions, and correctly identifies the true binder from a pool of 50 candidates in 29% of trials; this represents a substantial enrichment over random selection, yet remains far from reliable single-candidate identification. Structural consensus across models also discriminates binders from decoys, with pairwise accuracy reaching 73.1% when three or more models converge on a pose (DockQ>0.23, covering around 54.2% of pairs). Epitope localization remains unreliable: the best model contacts annotated epitope residues in only 62.8% of cases, indicating that binding discrimination does not establish accurate epitope localization. Our results suggest that current folding models provide useful but limited signal for antibody screening and should be complemented by orthogonal methods. We release POISK as a community resource for future benchmarking and model training.
The editor's rubric
- Leverage4 / 5
- A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).
- Magnitude2 / 5
- Solid incremental gain on a meaningful problem.
- Evidence3 / 5
- Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
- Novelty2 / 5
- A new combination of known ideas.
- Trajectory3 / 5
- A clear path to scale.
- Stakes2 / 5
- Benefits a professional community (practitioners, clinicians, engineers).
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); breadth (zero/few-shot); verification (experimental validation, code released); scale (scalable).
How the score was computed
Desks place stronger verdicts first, then use this score to order work within the same verdict.
- Merit
- 5.6 / 10
- Adjusted merit
- 4.6 / 10
- Attention
- 0%
- Freshness
- 88%