ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agents

By Yang, Bao, Hou +10Oct 9, 2026

Score█████░░░░░5.0

Key numbers

34.87% to 42.84%
55.71% and 56.25%

Caveats

Preprint; not yet peer reviewed.

VerdictCompetent work. Briefs at most.

Read the original PDF Code

Abstract

Coding agents are increasingly used to build programs from scratch as software engineers do rather than to resolve issues within a repository or to generate a repository from a given requirements document. However, scalable verifiable training environments that force agents to discover what is required and to decide how the system should be designed remain scarce. Moreover, existing agent workflows also cannot sustain long-horizon development in which the agent explores the reference program before implementation. To address these challenges, we introduce ProgramGym, a framework consisting of Program-Env and Program-Agent for scalable training environments of long-horizon program reproduction agents. Specifically, Program-Env constructs verifiable training environments from repositories through a feedback-driven loop that repairs runtime sandbox builds and the quality of black-box tests. Using this novel pipeline, we obtain 1,648 training environments spanning C, C++, Go, and Rust. Program-Agent combines a role-isolated workflow of analysis, implementation, and review with revision management for compilation checks and failure recovery. Experiments on ProgramBench show that both Program-Env and Program-Agent improve performance on program reproduction. Fine-tuning on trajectories with these constructed environments raises the performance of Qwen3.8-27B from 34.87% to 42.84%. Program-Agent further raises the base and SFT models to 55.71% and 56.25%, both exceeding some flagship models. We will release the ProgramGym to support further research on coding agents that build programs from scratch.

Chengyi Yang, Renxu Bao, Jue Hou, Zongpei Teng, Ziheng Wang, Yujie Lin, Chengsong Huang, Junnan Zhu, Delai Qiu, Yining Wang, Shengping Liu, Fei Long, Jinsong Su

The editor's rubric

Heuristic review

Dimension	Level	Weight	What that level means
Leverage	███░░ 3	24%	A method or resource many groups across the field will adopt within a year.
Magnitude	███░░ 3	18%	Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.
Evidence	██░░░ 2	14%	Limited: single setting, weak baselines, or an observational association presented as causal.
Novelty	███░░ 3	16%	A genuinely new approach to an open problem.
Trajectory	██░░░ 2	18%	Some room to improve with obvious engineering.
Stakes	██░░░ 2	10%	Benefits a professional community (practitioners, clinicians, engineers).

Editor’s rationale

Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); gains (relative gain, outperforms); novelty (alternative to status quo); scale (scalable); stakes (general AI).

How the score was computed

rank-2026-10-07

Score█████░░░░░5.0

Score = 10 × (65% × adjusted merit / 10 + 25% × attention + 10% × freshness)

Merit: 4.2 / 10; Weighted rubric, evidence-gated.
Adjusted merit: 4.1 / 10; Shrunk toward the desk prior by editor confidence (42%).
Attention: 75%; Citations, upvotes, points, mentions.
Freshness: 42%; Half-life decay since publication.

Hugging Face upvotes45 (reference 25, via hf-daily, Oct 12, 2026, 13:49 UTC)
GitHub stars1 (reference 250, via hf-daily, Oct 12, 2026, 13:49 UTC)

The record

Reviewed by heuristic-v6 on Oct 12, 2026, 01:23 UTC. Paper type: method.
Categories: cs.SE, cs.AI, cs.CL
BRIEF, No.2 in the Artificial Intelligence edition of October 12, 2026.