SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments

Computer-use agents (CUAs) are rapidly improving, yet synthesizing diverse and reliable software environments at scale remains challenging.

By Jin, Wang, Kang +4Oct 8, 2026

Score█████░░░░░5.2

Key numbers

41.39% to 56.97% on the
40.0% on SoftwareGym-Bench

Caveats

Preprint; not yet peer reviewed.

VerdictCompetent work. Briefs at most.

Read the original PDF Code

Abstract

Computer-use agents (CUAs) are rapidly improving, yet synthesizing diverse and reliable software environments at scale remains challenging. Native application environments require substantial construction and integration effort, which limits application coverage and increases the risk of application-specific overfitting. Browser-hosted environments provide limited interaction depth and workflow complexity. These limitations motivate SoftwareGym, an automated pipeline for synthesizing GUI software environments with diverse interfaces, executable functionality, and runtime-verified behavior. It generates structurally diverse designs, realizes them as runnable applications, and uses GUI evaluation with iterative repair to ensure functional reliability. Using SoftwareGym, we synthesize 182 applications across twelve categories and achieve a mean GUI task pass rate of 91.5%. We construct SoftwareGym-Bench, a 120-task evaluation suite spanning the same twelve categories, and use trajectories collected from these environments for CUA training. The resulting model improves Pass@3 from 41.39% to 56.97% on the strictly out-of-domain OSWorld benchmark and achieves 40.0% on SoftwareGym-Bench. These results demonstrate the potential of synthesized software environments as scalable and transferable supervision for CUA training.

Tian Jin, Wenhao Wang, Jing Kang, Haoting Shi, Rui Ye, Yijun Lu, Siheng Chen

The editor's rubric

Heuristic review

Dimension	Level	Weight	What that level means
Leverage	███░░ 3	24%	A method or resource many groups across the field will adopt within a year.
Magnitude	███░░ 3	18%	Large gain: roughly 2x, or a clear new state of the art on a hard, unsaturated problem.
Evidence	██░░░ 2	14%	Limited: single setting, weak baselines, or an observational association presented as causal.
Novelty	██░░░ 2	16%	A new combination of known ideas.
Trajectory	███░░ 3	18%	A clear path to scale.
Stakes	██░░░ 2	10%	Benefits a professional community (practitioners, clinicians, engineers).

Editor’s rationale

Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: breadth (many tasks); gains (relative gain); scale (scalable); stakes (general AI).

How the score was computed

rank-2026-10-07

Score█████░░░░░5.2

Score = 10 × (65% × adjusted merit / 10 + 25% × attention + 10% × freshness)

Merit: 4.3 / 10; Weighted rubric, evidence-gated.
Adjusted merit: 4.1 / 10; Shrunk toward the desk prior by editor confidence (38%).
Attention: 86%; Citations, upvotes, points, mentions.
Freshness: 35%; Half-life decay since publication.

Hugging Face upvotes79 (reference 25, via hf-daily, Oct 12, 2026, 13:49 UTC)
GitHub stars3 (reference 250, via hf-daily, Oct 12, 2026, 13:49 UTC)

The record

Reviewed by heuristic-v6 on Oct 12, 2026, 01:23 UTC. Paper type: empirical.
Categories: cs.MA, cs.SE
TOP, No.7 in the Artificial Intelligence edition of October 12, 2026.