Humanity’s List

AIarXiv

Ranked by formulaVerdict: Notable

HALO: Human-Grounded Granular Evaluation of Proactive Agents

Proactive agents must decide whether, when, and how far to assist without an explicit request.

By Pei, Wang, Liu +5

Score████░░░░░░3.7

Key numbers

  • 72.88% and 56.63% acceptability
  • 61.20% of rejected responses

Caveats

  • Preprint; not yet peer reviewed.

VerdictWorth a reader's time today.

Read the originalPDFCode

Abstract

Proactive agents must decide whether, when, and how far to assist without an explicit request. Existing evaluations often rely on automated scores or binary verdicts, offering limited human diagnosis of evidence use and intervention quality. To address these gaps, we introduce HALO, Human-Grounded GrAnular EvaLuation of PrOactive Agents. HALO-Arena covers 4 proactive need types, Dependency, Blockage, Risk, and Opportunity, across 10 everyday domains and 50 personal environments. Scenarios require agents to retrieve evidence from structured databases or unstructured documents before judging whether assistance is warranted and choosing its scope. On HALO-Arena, we construct HALO-8K, a dataset of 8,000 responses from 10 models across 400 scenarios, annotated by 20 experts for action level, acceptability verdicts, and human diagnosis. For rejected responses, human diagnosis covers timing, intensity, and other issues. Recorded reasoning tokens and retrieval rounds measure the effort behind these responses. The best proprietary and open-source models achieve 72.88% and 56.63% acceptability, respectively. Our central finding is that over-proactivity is the dominant observed failure mode, with Premature timing or Excessive intensity accounting for 61.20% of rejected responses. Further analyses yield complementary findings on the balance between intervention and restraint, evidence acquisition, and the calibration of timing and intensity.

Wenqi Pei, Xinyu Wang, Jinrui Liu, Cynthia Zhuge, Jie Sheng, Zihan Wang, Tenghao Huang, Muhao Chen

The editor's rubric

Heuristic review

Leverage4 / 5
A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).
24% of this desk’s rubric
Magnitude2 / 5
Solid incremental gain on a meaningful problem.
18% of this desk’s rubric
Evidence3 / 5
Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
14% of this desk’s rubric
Novelty2 / 5
A new combination of known ideas.
16% of this desk’s rubric
Trajectory2 / 5
Some room to improve with obvious engineering.
18% of this desk’s rubric
Stakes2 / 5
Benefits a professional community (practitioners, clinicians, engineers).
10% of this desk’s rubric

Editor’s rationale

Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); breadth (many tasks); verification (multiple benchmarks, code released); stakes (general AI).

How the score was computed

rank-2026-10-18

Score████░░░░░░3.7

Score = 10 × (65% × adjusted merit / 10 + 25% × attention + 10% × freshness)

Desks place stronger verdicts first, then use this score to order work within the same verdict.

Merit
5.2 / 10
Weighted rubric, evidence-gated.
Adjusted merit
4.5 / 10
Shrunk toward the desk prior by editor confidence (38%).
Attention
19%
Citations, upvotes, points, mentions.
Freshness
28%
Half-life decay since publication.
  • Citations0 (reference 15, via semantic-scholar, Nov 2, 2026, 02:05 UTC)
  • Influential citations0 (reference 3, via semantic-scholar, Nov 2, 2026, 02:05 UTC)
  • Hugging Face upvotes6 (reference 25, via hf-daily, Nov 2, 2026, 13:49 UTC)
  • GitHub stars1 (reference 250, via hf-daily, Nov 2, 2026, 13:49 UTC)

The record

  • Reviewed by heuristic-v10 on Oct 30, 2026, 13:36 UTC. Paper type: method.
  • Categories: cs.CL
  • BRIEF, No.1 in the Artificial Intelligence edition of November 2, 2026.