AIarXiv
Heuristic editor, no API keyVerdict: RoutineRepresentation-Space MMD for Diffusion Language Models
We introduce a post-training method for diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM.
VerdictCompetent work. Briefs at most.
Abstract
We introduce a post-training method for diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM. To estimate MMD, we retain contextual features at individual token positions, obtaining multiple observations per sequence from a single extractor pass. We optimize this objective using policy gradients for discrete models and direct differentiation through generated latents for continuous models. In both cases, computing the loss directly from these features enables efficient post-training without full sampling trajectories or jointly trained auxiliary models. Experiments show lower generative perplexity at comparable entropy on OpenWebText and better accuracy-computation trade-offs on GSM8K. On 16B DMax-LLaDA2.0 models with hybrid masked-uniform diffusion, we increase decoding parallelism with similar or higher accuracy on math and code benchmarks.
The editor's rubric
| Dimension | Level | Weight | What that level means |
|---|---|---|---|
| Leverage | ███░░ 3 | 24% | A method or resource many groups across the field will adopt within a year. |
| Magnitude | ██░░░ 2 | 18% | Solid incremental gain on a meaningful problem. |
| Evidence | ███░░ 3 | 14% | Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data. |
| Novelty | ██░░░ 2 | 16% | A new combination of known ideas. |
| Trajectory | ███░░ 3 | 18% | A clear path to scale. |
| Stakes | ██░░░ 2 | 10% | Benefits a professional community (practitioners, clinicians, engineers). |
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose); verification (multiple benchmarks, code released); scale (parallelizable, efficient); stakes (general AI).
How the score was computed
- Merit
- 5.1 / 10
- Adjusted merit
- 4.5 / 10
- Attention
- 44%
- Freshness
- 68%
- Hugging Face upvotes16 (reference 25, via hf-daily, Oct 7, 2026, 01:27 UTC)
- GitHub stars10 (reference 250, via hf-daily, Oct 7, 2026, 01:27 UTC)