去芜存菁：通过参数高效模块运算实现模型缺陷遗忘
Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation
AI总结：
本文提出 Ext-Sub 参数高效模块运算方法，通过结合专家与反专家 PEM，仅剔除缺陷能力而保留通用能力，从而提升 LLM 的真实性与去毒表现。
摘要：
大语言模型（LLM）已被广泛应用于各类应用，但众所周知存在与不真实性和毒性相关的问题。尽管参数高效模块（PEM）已证明其在赋予模型新技能方面的有效性，但利用 PEM 进行缺陷遗忘仍未得到充分探索。在本工作中，我们提出一种 PEM 运算方法，即“先提取后减除”（Extraction-before-Subtraction，Ext-Sub），通过整合“专家”PEM 与“反专家”PEM 来增强 LLM 的真实性与去毒效果。值得注意的是，即使是反专家 PEM 也具备有价值的能力，因为它们擅长生成捏造内容，而这需要语言建模和逻辑叙事能力。我们的方法并非简单地对参数取反，而是仅提取并消除反专家 PEM 中的缺陷能力，同时保留其通用能力。为评估该方法在真实性和去毒方面的有效性，我们在 LLM 上开展了大量实验，涵盖语言建模和数学推理等附加能力。实验结果表明，该方法有效提升了真实性和去毒效果，同时在很大程度上保留了 LLM 的基础能力。
英文摘要：
Large language models (LLMs) have been widely used in various applications but are known to suffer from issues related to untruthfulness and toxicity. While parameter-efficient modules (PEMs) have demonstrated their effectiveness in equipping models with new skills, leveraging PEMs for deficiency unlearning remains underexplored. In this work, we propose a PEMs operation approach, namely Extraction-before-Subtraction (Ext-Sub), to enhance the truthfulness and detoxification of LLMs through the integration of ``expert'' PEM and ``anti-expert'' PEM. Remarkably, even anti-expert PEM possess valuable capabilities due to their proficiency in generating fabricated content, which necessitates language modeling and logical narrative competence. Rather than merely negating the parameters, our approach involves extracting and eliminating solely the deficiency capability within anti-expert PEM while preserving the general capabilities. To evaluate the effectiveness of our approach in terms of truthfulness and detoxification, we conduct extensive experiments on LLMs, encompassing additional abilities such as language modeling and mathematical reasoning. Our empirical results demonstrate that our approach effectively improves truthfulness and detoxification, while largely preserving the fundamental abilities of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
