自主性税：防御训练破坏LLM智能体
The Autonomy Tax: Defense Training Breaks LLM Agents
- University of Southern California（南加州大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
揭示防御训练在提升LLM智能体安全性时，系统性地破坏其工具执行能力，导致任务失败率飙升，且无法有效防御复杂攻击。
AI中文摘要：
大型语言模型（LLM）智能体日益依赖外部工具（文件操作、API调用、数据库事务）来自主完成复杂的多步骤任务。实践者部署经过防御训练的模型，以防止通过恶意观察或检索内容操纵智能体行为的提示注入攻击。我们揭示了一个基本的\textbf{能力-对齐悖论}：旨在提高安全性的防御训练系统性地破坏了智能体的能力，同时未能阻止复杂的攻击。在97个智能体任务和1000个对抗性提示上，将防御模型与未防御基线进行比较，我们发现了多步骤智能体特有的三种系统性偏差。\textbf{智能体无能偏差}表现为立即的工具执行崩溃，模型在观察到任何外部内容之前就在良性任务上拒绝或生成无效操作。\textbf{级联放大偏差}导致早期失败通过重试循环传播，使防御模型在99%的任务中超时，而基线仅为13%。\textbf{触发偏差}导致矛盾的安全退化，防御模型的表现比未防御基线更差，而直接攻击以高概率绕过防御。根本原因分析表明，这些偏差源于捷径学习：模型过度拟合表面攻击模式而非语义威胁理解，这由防御效果在不同攻击类别上的极端方差所证明。我们的发现表明，当前的防御范式优化了单轮拒绝基准，同时使多步骤智能体从根本上不可靠，因此需要新的方法在对抗条件下保持工具执行能力。
英文摘要：
Large language model (LLM) agents increasingly rely on external tools (file operations, API calls, database transactions) to autonomously complete complex multi-step tasks. Practitioners deploy defense-trained models to protect against prompt injection attacks that manipulate agent behavior through malicious observations or retrieved content. We reveal a fundamental \textbf{capability-alignment paradox}: defense training designed to improve safety systematically destroys agent competence while failing to prevent sophisticated attacks. Evaluating defended models against undefended baselines across 97 agent tasks and 1,000 adversarial prompts, we uncover three systematic biases unique to multi-step agents. \textbf{Agent incompetence bias} manifests as immediate tool execution breakdown, with models refusing or generating invalid actions on benign tasks before observing any external content. \textbf{Cascade amplification bias} causes early failures to propagate through retry loops, pushing defended models to timeout on 99\% of tasks compared to 13\% for baselines. \textbf{Trigger bias} leads to paradoxical security degradation where defended models perform worse than undefended baselines while straightforward attacks bypass defenses at high rates. Root cause analysis reveals these biases stem from shortcut learning: models overfit to surface attack patterns rather than semantic threat understanding, evidenced by extreme variance in defense effectiveness across attack categories. Our findings demonstrate that current defense paradigms optimize for single-turn refusal benchmarks while rendering multi-step agents fundamentally unreliable, necessitating new approaches that preserve tool execution competence under adversarial conditions.
