辨别真伪：通过对比细化减少过度拒绝
Discern Truth from Falsehood: Reducing Over-Refusal via Contrastive Refinement
AI总结：
本文提出DCR方法，通过对比细化减少LLM的过度拒绝问题，同时保持安全性和通用能力。
摘要：
大型语言模型（LLMs）为了安全对齐时往往表现出过度拒绝的现象，即倾向于拒绝看似有毒或无害的提示，因为错误地将它们归类为有毒。这种行为损害了模型的有用性，并限制了在敏感或微妙情境下的可用性。尽管先前的工作提出了缓解策略，如数据增强和激活引导，但这些方法通常面临权衡：减少过度拒绝通常会降低模型拒绝真正有害内容的能力。我们主张，这种现象源于有毒和看似有毒提示对模型学习动态的模糊影响。为了解决这个问题，我们引入了一个前置对齐阶段，称为DCR：通过对比细化进行辨别。在理论和实证上，我们证明对比细化提高了LLM区分真正有毒提示和表面有毒提示的能力。在多样化的基准测试中，我们的方法有效减少了过度拒绝，同时保持对齐的安全优势。重要的是，它以最小的通用能力退化实现了这一点，为安全对齐提供了更系统和稳健的方向。
英文摘要：
Large language models (LLMs) aligned for safety often suffer from over-refusal, the tendency to reject seemingly toxic or benign prompts by misclassifying them as toxic. This behavior undermines models' helpfulness and restricts usability in sensitive or nuanced contexts. While prior work has proposed mitigation strategies such as data augmentation and activation steering, these approaches often face a trade-off: reducing over-refusal typically degrades the model's ability to reject genuinely harmful content. We argue that this issue arises from the ambiguous influence of toxic and seemingly toxic prompts on the model's learning dynamics. To address it, we introduce a preceding alignment stage, DCR: Discernment via Contrastive Refinement. Both theoretically and empirically, we demonstrate that contrastive refinement improves an LLM's capacity to distinguish truly toxic prompts from superficially toxic ones. Evaluation across diverse benchmarks shows that our method effectively reduces over-refusal while preserving the safety benefits of alignment. Importantly, it achieves this with minimal degradation of general capabilities, offering a more principled and robust direction for safety alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
