AIarXiv
Ranked by formulaVerdict: NotableSO-101 Bench: Accessible Spatial Manipulation
Low-cost arms such as the SO-101 have become the most common entry point to robot learning, yet no released policy performs spatially constrained, language-conditioned tasks on them out of the box, and existing SO-101…
Caveats
- Preprint; not yet peer reviewed.
VerdictWorth a reader's time today.
Abstract
Low-cost arms such as the SO-101 have become the most common entry point to robot learning, yet no released policy performs spatially constrained, language-conditioned tasks on them out of the box, and existing SO-101 benchmarks run only on physical hardware with simple tasks and few held-out objects. We introduce SO-101 Bench, a simulated benchmark for language-conditioned spatial manipulation built on an Isaac Lab digital twin of a real SO-101 setup. It has four tasks scored as five families, 138 objects in seen, unseen-instance and unseen-class splits, 916 evaluation episodes with subsets for held-out appearances, rotate-to-fit planning and occlusion memory, and 3,629 simulated demonstrations on which every learned policy is fine-tuned under an equal compute budget. The twin ranks policies as the real robot does: five policies trained without real data, four VLAs and an LLM agent, keep their exact ranking when the same weights run on the physical arm in an unseen environment with unseen objects (Spearman rho = 1.00, MMRV = 0.000), although simulation overestimates success. Across nine VLAs and world-action models, the best learned policies score near 61%, with spatial placement, unseen object classes and rotate-to-fit planning remaining open. An LLM agent that writes code and hands grasping to a learned policy scores 83.2%, and an ablation shows that this lead comes from the choice of harness and the choice of language model.
The editor's rubric
- Leverage4 / 5
- A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).
- Magnitude2 / 5
- Solid incremental gain on a meaningful problem.
- Evidence3 / 5
- Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
- Novelty2 / 5
- A new combination of known ideas.
- Trajectory3 / 5
- A clear path to scale.
- Stakes2 / 5
- Benefits a professional community (practitioners, clinicians, engineers).
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); breadth (many tasks); verification (multiple benchmarks, ablations, held-out test); scale (low cost); stakes (general AI).
How the score was computed
Desks place stronger verdicts first, then use this score to order work within the same verdict.
- Merit
- 5.6 / 10
- Adjusted merit
- 4.7 / 10
- Attention
- 10%
- Freshness
- 75%
- Hugging Face upvotes1 (reference 25, via hf-daily, Oct 28, 2026, 01:23 UTC)
- GitHub stars28 (reference 250, via hf-daily, Oct 28, 2026, 01:23 UTC)