arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.01042cs.LG
收藏

SafeSwitch:通过内部激活信号引导不安全的LLM行为

SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals

Siebel School of Computing(计算科学系);Data Science, University of Illinois Urbana-Champaign(数据科学,伊利诺伊大学厄巴纳-香槟分校);School of Business, Stevens Institute of Technology(商学院,史蒂文斯理工学院)

Peixuan Han, Cheng Qian, Xiusi Chen, Yuji Zhang, Heng Ji, Denghui Zhang

AI总结:

提出SafeSwitch框架,通过基于探测器的内部状态监视器检测有害意图并激活安全头,减少LLM有害输出约80%且仅微调不到6%参数,实现安全与效用的帕累托最优。

摘要:

大型语言模型(LLMs)在各种任务中展现出卓越的能力,但也因生成有害内容而带来风险。现有的安全机制虽然提升了模型安全性,但往往导致过度谨慎的行为,且未能充分利用LLMs的内部认知过程。受人类反思性思维能力的启发,我们首先表明LLMs同样能在其内部状态中对安全性进行内部评估。基于这一洞察,我们提出了SafeSwitch,这是一个动态框架,通过利用基于探测器的内部状态监视器来主动检测有害意图,并仅在必要时激活导致更安全、更保守响应的安全头,从而调节不安全输出。SafeSwitch在有害查询上将有害输出减少了约80%,同时保持了强大的实用性,在多种方法中达到了帕累托最优。我们的方法在提供更具信息量、具备上下文感知的拒绝方面也优于传统方法,并且在仅微调不到6%原始参数的情况下实现了这些收益。SafeSwitch展示了大型语言模型在安全性方面的自我意识和反思能力,为更细致、更有效的安全控制提供了一种有前景的方法。本工作的代码可在https://github.com/Hanpx20/SafeSwitch获取。

英文摘要:

Large language models (LLMs) exhibit exceptional capabilities across various tasks but also pose risks by generating harmful content. Existing safety mechanisms, while improving model safety, often lead to overly cautious behavior and fail to fully leverage LLMs' internal cognitive processes. Inspired by humans' reflective thinking capability, we first show that LLMs can similarly perform internal assessments about safety in their internal states. Building on this insight, we propose SafeSwitch, a dynamic framework that regulates unsafe outputs by utilizing the prober-based internal state monitor that actively detects harmful intentions, and activates a safety head that leads to safer and more conservative responses only when necessary. SafeSwitch reduces harmful outputs by approximately 80% on harmful queries while maintaining strong utility, reaching a Pareto optimal among several methods. Our method is also advantageous over traditional methods in offering more informative, context-aware refusals, and achieves these benefits while only tuning less than 6% of the original parameters. SafeSwitch demonstrates large language models' capacity for self-awareness and reflection regarding safety, offering a promising approach to more nuanced and effective safety controls. Codes for this work are available at https://github.com/Hanpx20/SafeSwitch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑