多语言语言模型中选择性知识的跨语言遗忘
Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models
AI总结：
本文针对多语言语言模型中单语言遗忘无法跨语言迁移的问题，提出一种自适应遗忘方案，通过语言相关权重实现选择性跨语言知识擦除，在保持整体性能的同时提升安全性。
摘要：
预训练语言模型记忆了大量信息，包括私人和受版权保护的数据，引发了重大的安全担忧。在排除敏感数据后重新训练这些模型成本极其高昂，这使得机器遗忘成为一种可行且具有成本效益的替代方案。以往的研究主要关注单语言模型的机器遗忘，但我们发现，在一种语言中进行的遗忘并不一定会迁移到其他语言。这一脆弱性使模型容易受到低资源语言攻击，即敏感信息在较少占主导地位的语言中仍然可以被访问。本文提出了一种面向多语言语言模型的机器遗忘开创性方法，在保持整体性能的同时，选择性地跨不同语言擦除信息。具体而言，我们的方法采用了一种自适应遗忘方案，该方案分配与语言相关的权重，以应对多语言语言模型在不同语言上的性能差异。实证结果表明，与现有的遗忘基线相比，我们的框架具有有效性，为安全且具有适应性的多语言语言模型设立了新的标准。
英文摘要：
Pretrained language models memorize vast amounts of information, including private and copyrighted data, raising significant safety concerns. Retraining these models after excluding sensitive data is prohibitively expensive, making machine unlearning a viable, cost-effective alternative. Previous research has focused on machine unlearning for monolingual models, but we find that unlearning in one language does not necessarily transfer to others. This vulnerability makes models susceptible to low-resource language attacks, where sensitive information remains accessible in less dominant languages. This paper presents a pioneering approach to machine unlearning for multilingual language models, selectively erasing information across different languages while maintaining overall performance. Specifically, our method employs an adaptive unlearning scheme that assigns language-dependent weights to address different language performances of multilingual language models. Empirical results demonstrate the effectiveness of our framework compared to existing unlearning baselines, setting a new standard for secure and adaptable multilingual language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
