AIarXiv
Ranked by formulaVerdict: NotableSecPO: Principled Adversarial Training for Prompt Injection Security
Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation.
Key numbers
- 14.5× across three SoTA adaptive
- 4.6× compared with using DPO
- 100% under adaptive prompt injections
Caveats
- Preprint; not yet peer reviewed.
VerdictWorth a reader's time today.
Abstract
Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation. The existing best-defended LLM, Meta-SecAlign, is trained with the DPO loss but still suffers from attack success rates (ASRs) approaching 100% under adaptive prompt injections. One reason is that DPO anchors the model around the undefended reference model, which is insecure. We show an idealized reference model---an injection-invariant, utility-preserving oracle---can nevertheless be approximated, as its output distribution in training is interestingly accessible. This new observation motivates a theoretically principled SecPO loss. Another Meta-SecAlign shortcoming is that it trains on weak simulated attacks. We implement an efficient adversarial training (AT) pipeline that synthesizes strong injected inputs (best@240), and train the model on them. Our AT reduces ASRs by 1.5-14.5× across three SoTA adaptive attacks, compared with Meta-SecAlign; the SecPO loss reduces them by a further 2.4-4.6× compared with using DPO loss. In agentic workflows, our SecPO + AT Qwen3.6-27B is much more secure than Meta-SecAlign: 8.8% vs. 78.8% AgentDojo ASR@800 under the AttackerMovesSecond attack. Moreover, our defended model shows no noticeable reduction in benign utility in nine benchmarks covering agentic tool calling, instruction following, science, and math. We release our href//github.com/Sizhe-Chen/SecPOcode, $\href{//huggingface.co/datasets/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO-trainset}{training data}$ (with optimized injections from AT), and $\href{//huggingface.co/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO}{model}$.
The editor's rubric
- Leverage4 / 5
- A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).
- Magnitude4 / 5
- A qualitative jump: a capability or regime that did not exist before.
- Evidence3 / 5
- Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
- Novelty2 / 5
- A new combination of known ideas.
- Trajectory3 / 5
- A clear path to scale.
- Stakes2 / 5
- Benefits a professional community (practitioners, clinicians, engineers).
Editor’s rationale
Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we report); breadth (many tasks); gains (x-fold, versus baseline, state of the art); verification (multiple benchmarks, code released); scale (efficient); stakes (general AI).
How the score was computed
Desks place stronger verdicts first, then use this score to order work within the same verdict.
- Merit
- 6.3 / 10
- Adjusted merit
- 5.1 / 10
- Attention
- 0%
- Freshness
- 45%