防止灾难性遗忘：面向更安全语言模型微调的行为感知采样
Preventing Catastrophic Forgetting: Behavior-Aware Sampling for Safer Language Model Fine-Tuning
AI总结：
针对微调中的灾难性遗忘，提出行为感知采样框架，基于指令响应行为与语义多样性选择安全示例，仅用0.5%额外数据将有害性降低41%。
摘要：
大型语言模型在良性数据上进行微调时，常常会丢失先前对齐的安全行为，这一现象被称为灾难性遗忘。先前的研究表明，添加随机的安全示例可以缓解这一影响，但尚不清楚哪些示例最为有效。我们提出了一种行为感知采样框架，该框架基于两个互补因素选择安全示例：指令-响应行为（例如，拒绝与服从）以及跨危害类别的语义多样性。系统性评估表明，该方法在保持有用性的同时显著减少了有害输出，仅增加0.5%的额外训练数据即可实现高达41%的有害性降低。这些结果凸显了有针对性的数据选择如何在规模上提高微调的安全性和效率。
英文摘要：
Large language models often lose previously aligned safety behaviors when fine-tuned on benign data, a phenomenon known as catastrophic forgetting. Prior work shows that adding random safety examples can mitigate this effect, but it remains unclear which examples are most effective. We propose a behavior-aware sampling framework that selects safety examples based on two complementary factors: instruction-response behavior (e.g., refusal versus compliance) and semantic diversity across harm categories. Systematic evaluation shows that this approach substantially reduces harmful outputs while maintaining helpfulness, achieving up to a 41% reduction in harmfulness with only 0.5% additional training data. These results highlight how targeted data selection can improve the safety and efficiency of fine-tuning at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
