用对比学习支付对齐税
Paying Alignment Tax with Contrastive Learning
AI总结：
针对现有去偏方法导致模型能力退化的问题，提出对比学习框架，通过正负例和动态损失缩放，同时提升毒性降低与忠实性，避免对齐税。
摘要：
当前的去偏方法常常导致模型能力下降，如事实准确性和知识保留能力受损。通过在多个基准上的系统评估，我们证明现有的去偏方法面临根本性的权衡，尤其在较小的模型中，导致真实性降低、知识丢失或输出难以理解。为解决这些局限，我们提出一个对比学习框架，通过精心构造的正例和负例进行学习。我们的方法引入对比计算和动态损失缩放，以在偏见缓解与忠实性保持之间取得平衡。跨多个模型规模的实验结果表明，我们的方法在毒性降低和忠实性保持两方面均取得了显著改进。最重要的是，我们表明该框架是首个同时持续改进这两个指标的方法，避免了现有方法所特有的能力退化。这些结果表明，通过对比学习显式建模正负例可能是减少语言模型去偏中对齐税的一个有前景的方向。
英文摘要：
Current debiasing approaches often result a degradation in model capabilities such as factual accuracy and knowledge retention. Through systematic evaluation across multiple benchmarks, we demonstrate that existing debiasing methods face fundamental trade-offs, particularly in smaller models, leading to reduced truthfulness, knowledge loss, or unintelligible outputs. To address these limitations, we propose a contrastive learning framework that learns through carefully constructed positive and negative examples. Our approach introduces contrast computation and dynamic loss scaling to balance bias mitigation with faithfulness preservation. Experimental results across multiple model scales demonstrate that our method achieves substantial improvements in both toxicity reduction and faithfulness preservation. Most importantly, we show that our framework is the first to consistently improve both metrics simultaneously, avoiding the capability degradation characteristic of existing approaches. These results suggest that explicit modeling of both positive and negative examples through contrastive learning could be a promising direction for reducing the alignment tax in language model debiasing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
