arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.17633cs.CLcs.AI
收藏

大语言模型中的过度拒答问题探究

Navigating the OverKill in Large Language Models

Fudan University(复旦大学);University of California, Santa Barbara(加州大学圣巴巴拉分校);Shanghai AI Laboratory(上海人工智能实验室)

Chenyu Shi, Xiao Wang, Qiming Ge, Songyang Gao, Xianjun Yang, Tao Gui, Qi Zhang, Xuanjing Huang, Xun Zhao, Dahua Lin

AI总结:

针对大语言模型因对有害词过度关注、安全提示加剧而产生的良性查询过度拒答问题,提出无需训练、模型无关的自对比解码方法,通过对比有无安全强调的输出分布弱化过度关注,实现拒答率平均降20%且几乎不影响安全性。

摘要:

大语言模型经过精细对齐,以同时实现有用性与无害性。然而近期研究指出存在一种潜在的过度拒答(OverKill)现象,即模型可能会拒绝回答良性查询。本文通过探究模型如何处理和判定查询的安全性,研究了导致过度拒答的因素。研究结果表明,模型内部存在捷径机制,导致其对“kill”这类有害词汇产生过度关注,而强调安全性的提示词会加剧过度拒答问题。基于这些发现,我们提出了自对比解码(Self-Contrastive Decoding,Self-CD)——一种无需训练、模型无关的策略——来缓解这一现象。我们首先通过放大模型在响应包含或不包含安全性强调的系统提示时的输出分布差异,提取出这类过度关注。随后我们通过对比解码弱化模型的过度关注,从而确定最终的下一词元预测。实验结果表明,我们的方法使拒答率平均降低了20%,同时几乎对安全性没有影响。

英文摘要:

Large language models are meticulously aligned to be both helpful and harmless. However, recent research points to a potential overkill which means models may refuse to answer benign queries. In this paper, we investigate the factors for overkill by exploring how models handle and determine the safety of queries. Our findings reveal the presence of shortcuts within models, leading to an over-attention of harmful words like 'kill' and prompts emphasizing safety will exacerbate overkill. Based on these insights, we introduce Self-Contrastive Decoding (Self-CD), a training-free and model-agnostic strategy, to alleviate this phenomenon. We first extract such over-attention by amplifying the difference in the model's output distributions when responding to system prompts that either include or omit an emphasis on safety. Then we determine the final next-token predictions by downplaying the over-attention from the model via contrastive decoding. Empirical results indicate that our method has achieved an average reduction of the refusal rate by 20\% while having almost no impact on safety.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑