AgentProofBench: Benchmarking Reusable Verifiers for Web Agents

Reusable Web-agent verification requires a rule to follow changing requirements and support subsequent execution.

By Ning, Li, Lu +1Oct 25, 2026

Score████░░░░░░4.0

Caveats

Preprint; not yet peer reviewed.

VerdictWorth a reader's time today.

Read the original PDF Code

Abstract

Reusable Web-agent verification requires a rule to follow changing requirements and support subsequent execution. We introduce AgentProofBench, which connects paired tests of a frozen acceptance rule to execution from identical saved openings. The resource spans 97 tasks, 79 families, and four Web services. Experiments with DeepSeek Flash/Pro establish two linked measurements. Across 48 families, complete-panel and decision-replay tests retain 82/96 Flash and 79/96 Pro source programs; paired reuse-rebuild BA intervals are [-5.47, 1.57] and [-10.16, 0.00] percentage points. Across 22 execution families, deployment-state panel correctness separates program assignments whose two saved openings are both delivered correctly: 32/33 assignments with error-free panels, versus 1/11 with panel errors. All reference-defined program-gate regressions and withheld deliveries occur in the latter group. Neutral continuation attains the highest correct delivery under the tested generic feedback, providing a concrete baseline for measuring the value added by a gate. Released instructions, states, programs, reference rules, and trajectories connect each test failure to its execution consequences. AgentProofBench makes reusable verification assessable at both the requirement and policy levels.

Jingjie Ning, Xueqi Li, Yi Lu, Yibo Kong

The editor's rubric

Heuristic review

Leverage4 / 5: A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).; 24% of this desk’s rubric
Magnitude2 / 5: Solid incremental gain on a meaningful problem.; 18% of this desk’s rubric
Evidence3 / 5: Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.; 14% of this desk’s rubric
Novelty2 / 5: A new combination of known ideas.; 16% of this desk’s rubric
Trajectory2 / 5: Some room to improve with obvious engineering.; 18% of this desk’s rubric
Stakes2 / 5: Benefits a professional community (practitioners, clinicians, engineers).; 10% of this desk’s rubric

Dimension	Level	Weight	What that level means
Leverage	████░4/5	24%	A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).
Magnitude	██░░░2/5	18%	Solid incremental gain on a meaningful problem.
Evidence	███░░3/5	14%	Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
Novelty	██░░░2/5	16%	A new combination of known ideas.
Trajectory	██░░░2/5	18%	Some room to improve with obvious engineering.
Stakes	██░░░2/5	10%	Benefits a professional community (practitioners, clinicians, engineers).

Editor’s rationale

Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); breadth (many tasks); verification (multiple benchmarks); stakes (general AI).

How the score was computed

rank-2026-10-18

Score████░░░░░░4.0

Score = 10 × (65% × adjusted merit / 10 + 25% × attention + 10% × freshness)

Desks place stronger verdicts first, then use this score to order work within the same verdict.

Merit: 5.2 / 10; Weighted rubric, evidence-gated.
Adjusted merit: 4.5 / 10; Shrunk toward the desk prior by editor confidence (38%).
Attention: 16%; Citations, upvotes, points, mentions.
Freshness: 64%; Half-life decay since publication.

Hugging Face upvotes5 (reference 25, via hf-daily, Oct 27, 2026, 13:49 UTC)
GitHub stars0 (reference 250, via hf-daily, Oct 27, 2026, 13:49 UTC)

The record

Reviewed by heuristic-v10 on Oct 27, 2026, 13:49 UTC. Paper type: method.
Categories: cs.MA, cs.AI, cs.SE
BRIEF, No.7 in the Artificial Intelligence edition of October 27, 2026.