拒绝-顺从权衡：大型语言模型的大规模安全行为审计
The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models
AI总结：
本研究通过调整组合方法隔离模型敏感性与数据集毒性混淆，审计了21个开源权重LLM在四个安全基准上的拒绝与顺从失败模式，发现模型采用不同校准策略、人口保护不平等以及拒绝与顺从倾向在模型家族内稳定。
摘要：
拒绝率是LLM安全性的一个不良代理指标，即模型可能过度拒绝良性提示，同时仍顺从有害提示。我们审计了21个开源权重LLM在四个安全基准（OR-Bench、XSTest、ToxiGen、BOLD）上的两种失败模式，使用组合调整来隔离模型敏感性与数据集毒性混淆。我们报告三个发现。首先，模型采用根本不同的校准策略：保守生态系统（如Llama）以过度拒绝为代价抑制不安全输出，而宽松生态系统（如DeepSeek和Qwen）保持有用性但容忍更高的有害顺从。其次，人口保护不平等：模型过度保护突出的种族和宗教群体，经常拒绝甚至关于它们的良性提示，而对针对残疾的攻击提供显著较弱的保护。第三，拒绝和顺从倾向在模型家族内跨代和规模稳定，表明后训练目标比架构更能塑造安全行为。我们的结果呼吁进行联合、人口意识感知和多评判者的安全评估。
英文摘要：
Refusal rates are a poor proxy for LLM safety, i.e., a model may over-refuse benign prompts while still complying with harmful ones. We audit both failure modes across 21 open-weight LLMs on four safety benchmarks (OR-Bench, XSTest, ToxiGen, BOLD), using a composition adjustment to isolate model sensitivity from dataset toxicity confounds. We report three findings. First, models adopt fundamentally different calibration strategies: conservative ecosystems such as Llama suppress unsafe outputs at the cost of elevated over-refusals, while permissive ecosystems such as DeepSeek and Qwen preserve helpfulness but tolerate higher harmful compliance. Second, demographic protection is unequal: models over-protect prominent racial and religious groups, frequently refusing even benign prompts about them, while providing substantially weaker protection against disability-targeted attacks. Third, refusal and compliance tendencies are stable within model families across generations and scales, suggesting that post-training objectives shape safety behavior more than architecture. Our results call for joint, demographically-aware, and multi-judge safety evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
