分析和编辑后门语言模型的内部机制
Analyzing And Editing Inner Mechanisms Of Backdoored Language Models
AI总结：
本文研究后门语言模型的内部机制，发现早期层 MLP 模块与初始嵌入投影对后门行为最关键，并提出基于主成分的低秩替换方法 PCP 消融，用于移除、插入和修改后门，同时通过在微调时局部约束模块提升模型对数据投毒的后门鲁棒性。
摘要：
数据集投毒是对大型语言模型的一种潜在安全威胁，可能导致模型被植入后门。关于后门语言模型的内部机制，以及它们如何处理触发输入（例如在切换到有毒语言时），目前尚未有清晰的描述。在这项工作中，我们研究了基于 Transformer 的后门语言模型的内部表征，并确定早期层的 MLP 模块与初始嵌入投影相结合对后门机制最为重要。我们利用这一认识，通过工程化替换来移除、插入和修改后门机制，这些替换将 MLP 模块的输出缩减为后门机制所必需的核心部分。为此，我们引入了 PCP 消融方法，即基于 Transformer 模块激活的主成分，用低秩矩阵替换这些模块。我们在后门玩具模型、后门大型模型以及无后门的开源模型上展示了结果。我们证明，通过在可能被投毒的数据集上进行微调时局部约束单个模块，可以提高大型语言模型对后门的鲁棒性。\n触发警告：冒犯性语言。
英文摘要：
Poisoning of data sets is a potential security threat to large language models that can lead to backdoored models. A description of the internal mechanisms of backdoored language models and how they process trigger inputs, e.g., when switching to toxic language, has yet to be found. In this work, we study the internal representations of transformer-based backdoored language models and determine early-layer MLP modules as most important for the backdoor mechanism in combination with the initial embedding projection. We use this knowledge to remove, insert, and modify backdoor mechanisms with engineered replacements that reduce the MLP module outputs to essentials for the backdoor mechanism. To this end, we introduce PCP ablation, where we replace transformer modules with low-rank matrices based on the principal components of their activations. We demonstrate our results on backdoored toy, backdoored large, and non-backdoored open-source models. We show that we can improve the backdoor robustness of large language models by locally constraining individual modules during fine-tuning on potentially poisonous data sets. Trigger warning: Offensive language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
