AIarXiv
Heuristic editor, no API keyVerdict: RoutineLearning Meta-Skills for Agent Harness Design in Test-Time AI4AI
Agent performance depends on both reasoning ability and the environment in which it acts.
VerdictCompetent work. Briefs at most.
Abstract
Agent performance depends on both reasoning ability and the environment in which it acts. We study test-time AI-for-AI, asking how a Builder can learn to construct better execution environments for a Target while both models' weights remain fixed. To make the Builder's experience reusable, we introduce Meta-Skill: principles specifying when support is needed and what resources to provide. The Builder learns these principles from Target's execution feedback on the development set, then uses the frozen skill bank to construct harnesses for unseen tasks. Across Harness-Bench and NewtonBench, full-bank meta-skills improve macro-average performance by 8.95 percentage points over no-skill construction, and 12.02 points over direct delivery of the same bank to the Target. These results highlight the value of translating experience into executable support. Gains when the same model serves both roles further suggest a path to system level self-improvement through learning to build better environments.
The editor's rubric
| Dimension | Level | Weight | What that level means |
|---|---|---|---|
| Leverage | ███░░ 3 | 24% | A method or resource many groups across the field will adopt within a year. |
| Magnitude | ███░░ 3 | 18% | Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem. |
| Evidence | ███░░ 3 | 14% | Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data. |
| Novelty | ██░░░ 2 | 16% | A new combination of known ideas. |
| Trajectory | ██░░░ 2 | 18% | Some room to improve with obvious engineering. |
| Stakes | ██░░░ 2 | 10% | Benefits a professional community (practitioners, clinicians, engineers). |
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); gains (relative gain); verification (held-out test); stakes (general AI).
How the score was computed
- Merit
- 5.1 / 10
- Adjusted merit
- 4.4 / 10
- Attention
- 87%
- Freshness
- 30%
- Citations0 (reference 15, via semantic-scholar, Oct 3, 2026, 13:49 UTC)
- Influential citations0 (reference 3, via semantic-scholar, Oct 3, 2026, 13:49 UTC)
- Hugging Face upvotes80 (reference 25, via hf-daily, Oct 3, 2026, 13:49 UTC)
- GitHub stars6 (reference 250, via hf-daily, Oct 3, 2026, 13:49 UTC)