AIarXiv
Ranked by formulaVerdict: RoutineRecovering Reliable Supervision via Critique-Guided On-Policy Distillation
On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes.
Caveats
- Preprint; not yet peer reviewed.
VerdictCompetent work. Briefs at most.
Abstract
On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the recoverability of the teacher and show it fails on incorrect rollouts: the teacher follows student errors, although it recognizes them. We propose CriticOPD, using error recognition to restore recoverability. The teacher critiques an incorrect rollout, allowing the student to rewrite its solution following the error. The resulting output, including the critique as privileged information, is scored by the teacher while the student uses the revision without the critique for training. On two Qwen3 students and six mathematical reasoning benchmarks, CriticOPD improves OPD, with the largest gains on competition problems.
The editor's rubric
| Dimension | Level | Weight | What that level means |
|---|---|---|---|
| Leverage | ███░░ 3 | 24% | A method or resource many groups across the field will adopt within a year. |
| Magnitude | ██░░░ 2 | 18% | Solid incremental gain on a meaningful problem. |
| Evidence | ██░░░ 2 | 14% | Limited: single setting, weak baselines, or an observational association presented as causal. |
| Novelty | ██░░░ 2 | 16% | A new combination of known ideas. |
| Trajectory | ██░░░ 2 | 18% | Some room to improve with obvious engineering. |
| Stakes | ██░░░ 2 | 10% | Benefits a professional community (practitioners, clinicians, engineers). |
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose, named contribution); verification (code released); stakes (general AI).
How the score was computed
- Merit
- 3.7 / 10
- Adjusted merit
- 3.9 / 10
- Attention
- 82%
- Freshness
- 79%
- Hugging Face upvotes61 (reference 25, via hf-daily, Oct 15, 2026, 13:49 UTC)
- GitHub stars1 (reference 250, via hf-daily, Oct 15, 2026, 13:49 UTC)