Deep Search in Real-World Web Environments

Despite substantial progress in Deep Search agents, their ability to conduct end-to-end research in real-world open Web environments remains insufficiently evaluated.

By Yu, Chen, Huang +22Oct 19, 2026

Score██████░░░░5.7

Key numbers

39.0% accuracy
21.1% accuracy

Caveats

Preprint; not yet peer reviewed.

VerdictWorth a reader's time today.

Read the original PDF Code

Abstract

Despite substantial progress in Deep Search agents, their ability to conduct end-to-end research in real-world open Web environments remains insufficiently evaluated. Unlike static retrieval settings, information on the real-world Web is distributed across diverse sources, modalities, and heterogeneous formats, often involving complex interaction mechanisms that require agents to autonomously search for, access, and integrate relevant evidence during execution. To address this gap, we introduce the OpenDS task, designed to evaluate agents' ability to perform multi-step deep search in the live, open Web. This task requires agents to autonomously explore the real Web, discover and connect relevant evidence from multiple independent sources through Web search, browser interaction, and evidence processing, and ultimately determine a unique factual answer. Building on this task formulation, we construct OpenDS-World, a benchmark comprising 200 challenging tasks spanning diverse information modalities, heterogeneous file formats, and interactive access requirements, with independent human verification to ensure evidence validity and answer uniqueness. We evaluate 12 frontier models under a unified tool environment and execution protocol. The best-performing model achieves 39.0% accuracy, while human evaluation across 1,000 independent attempts yields 21.1% accuracy. Further analyses of tool usage, search cost, termination behavior, and failure recovery reveal substantial differences in research strategies across models, as well as persistent challenges in tool coordination and long-horizon search management. OpenDS-World provides a unified evaluation platform for systematically assessing and advancing agents' autonomous deep search capabilities in real-world open Web environments.

Xiaomin Yu, Hongliang Chen, Zhe Huang, Shilin Yan, Shao-Rong Su, Xuqiao Ma, Feiyang Wang, Jiayu Chen, Jiwei Fang, Yu Yan, Yini Huang, Chen Chen, Ruoxi Cheng, Yuzhi He, Yuhui Zhang, Xiaoxing Hu, Chen Liu, Jianbo Lin, Yijiang Li, Xiaobin Hu, Menglong Li, Zhuoran Lu, Weishi Wang, Zhongqi Yue, Chengwei Qin

The editor's rubric

Heuristic review

Leverage4 / 5: A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).; 24% of this desk’s rubric
Magnitude2 / 5: Solid incremental gain on a meaningful problem.; 18% of this desk’s rubric
Evidence3 / 5: Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.; 14% of this desk’s rubric
Novelty2 / 5: A new combination of known ideas.; 16% of this desk’s rubric
Trajectory3 / 5: A clear path to scale.; 18% of this desk’s rubric
Stakes2 / 5: Benefits a professional community (practitioners, clinicians, engineers).; 10% of this desk’s rubric

Dimension	Level	Weight	What that level means
Leverage	████░ 4	24%	A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).
Magnitude	██░░░ 2	18%	Solid incremental gain on a meaningful problem.
Evidence	███░░ 3	14%	Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
Novelty	██░░░ 2	16%	A new combination of known ideas.
Trajectory	███░░ 3	18%	A clear path to scale.
Stakes	██░░░ 2	10%	Benefits a professional community (practitioners, clinicians, engineers).

Editor’s rationale

Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose, new method); breadth (many tasks, programmable); verification (multiple benchmarks, human evaluation, independent replication); stakes (general AI).

How the score was computed

rank-2026-10-18

Score██████░░░░5.7

Score = 10 × (65% × adjusted merit / 10 + 25% × attention + 10% × freshness)

Desks place stronger verdicts first, then use this score to order work within the same verdict.

Merit: 5.6 / 10; Weighted rubric, evidence-gated.
Adjusted merit: 4.7 / 10; Shrunk toward the desk prior by editor confidence (46%).
Attention: 90%; Citations, upvotes, points, mentions.
Freshness: 35%; Half-life decay since publication.

Hugging Face upvotes145 (reference 25, via hf-daily, Oct 23, 2026, 13:24 UTC)
GitHub stars14 (reference 250, via hf-daily, Oct 23, 2026, 13:24 UTC)

The record

Reviewed by heuristic-v7 on Oct 23, 2026, 02:53 UTC. Paper type: method.
Categories: cs.IR, cs.MM, cs.SE
TOP, No.2 in the Artificial Intelligence edition of October 23, 2026.