arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.11096cs.CLcs.AI
收藏

推理模型中的安全恢复仅需几步早期引导步骤

Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away

University of Maryland, College Park(马里兰大学哥伦比亚学院);IIT, Bombay(孟买印度理工学院);University of Central Florida(佛罗里达中央大学)

Soumya Suvra Ghosal, Souradip Chakraborty, Vaibhav Singh, Furong Huang, Dinesh Manocha, Amrit Singh Bedi

AI总结:

SafeThink通过在推理早期干预减少攻击成功率,提升推理模型的安全性

摘要:

基于强化学习(RL)的后训练方法(如GRPO)能够提升多模态大规模推理模型(MLRMs)的推理能力。但最近的证据表明,这种方法同时会降低安全性对齐并增加逃脱成功率。我们提出了SafeThink,一种轻量级的推理时间防御机制,将安全性恢复视为满足条件的约束而非最大化目标。SafeThink通过安全奖励模型监控演化的推理轨迹,并在安全阈值被违反时有条件地注入优化的简短纠正前缀(

英文摘要:

Reinforcement learning (RL) based post-training for explicit chain-of-thought (e.g., GRPO) improves the reasoning ability of multimodal large-scale reasoning models (MLRMs). But recent evidence shows that it can simultaneously degrade safety alignment and increase jailbreak success rates. We propose SafeThink, a lightweight inference-time defense that treats safety recovery as a satisficing constraint rather than a maximization objective. SafeThink monitors the evolving reasoning trace with a safety reward model and conditionally injects an optimized short corrective prefix ("Wait, think safely") only when the safety threshold is violated. In our evaluations across six open-source MLRMs and four jailbreak benchmarks (JailbreakV-28K, Hades, FigStep, and MM-SafetyBench), SafeThink reduces attack success rates by 30-60% (e.g., LlamaV-o1: 63.33% to 5.74% on JailbreakV-28K, R1-Onevision: 69.07% to 5.65% on Hades) while preserving reasoning performance (MathVista accuracy: 65.20% to 65.00%). A key empirical finding from our experiments is that safety recovery is often only a few steering steps away: intervening in the first 1-3 reasoning steps typically suffices to redirect the full generation toward safe completions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑