并非所有转折都重要：多轮 jailbreak 的信用分配
Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreaking
AI总结：
本文提出 TRACE 框架，通过区分多轮 jailbreak 中不同回合的贡献，提升攻击效果和防御安全。
摘要：
在多轮对话中部署 LLM 会引发将有害意图分散到看似无害回合的 jailbreak 攻击。最近的基于训练的多轮 jailbreak 方法通过交互反馈学习长周期攻击策略，但通常依赖于粗粒度轨迹级结果信号，这些信号均匀广播给每个回合。然而，我们发现多轮 jailbreaking 中的回合贡献是非均匀、相位依赖和目标特定的。这种粗粒度结果监督导致了信用分配问题，使得成功轨迹中冗余回合被过度奖励，而失败轨迹中的有用中间回合被低估。为此，我们提出 TRACE，一种基于强化学习（RL）的多轮 jailbreak 的回合意识信用分配框架。对于成功轨迹，TRACE 通过留出一个回合的语义遮蔽来估计回合级贡献；对于失败轨迹，TRACE 根据提示的有害性和语义相关性分配惩罚，并额外添加一个本地拒绝意识惩罚。此外，我们重用攻击侧的信用信号用于多轮防御对齐。在开源和闭源目标上的大量实验表明，TRACE 在有效性、可转移性和效率方面均表现出色，相较于最强的 RL 基线，攻击成功率提高了约 25%，同时在用于防御对齐时也改善了安全与效用的平衡。
英文摘要：
Deploying LLMs in multi-turn dialogues facilitates jailbreak attacks that distribute harmful intent across seemingly benign turns. Recent training-based multi-turn jailbreak methods learn long-horizon attack strategies from interaction feedback, but often rely on coarse trajectory-level outcome signals that broadcast uniformly to every turn. However, we find that turn-level contributions in multi-turn jailbreaking are non-uniform, phase-dependent, and target-specific. Such coarse outcome supervision induces a credit assignment problem, leading to over-rewarding redundant turns in successful trajectories and under-crediting useful intermediate turns in failed ones. To address this, we propose TRACE, a turn-aware credit assignment framework for reinforcement learning (RL)-based multi-turn jailbreaking. For successful trajectories, TRACE estimates turn-level contributions via leave-one-turn-out semantic masking; for failed ones, TRACE assigns penalties based on prompt harmfulness and semantic relevance, with an additional local refusal-aware penalty. Furthermore, we reuse the attack-side credit signal for multi-turn defense alignment. Extensive experiments on open-source and closed-source targets show that TRACE achieves strong overall performance in effectiveness, transferability, and efficiency, yielding about a 25% relative improvement in attack success rate over the strongest RL baseline while also improving the safety-utility balance when reused for defense alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
