arXiv 2406.115662024-06-17cs.CL

MEMLA：利用神经元掩码低秩适配增强多语言知识编辑

MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptation

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）

Jiakuan Xie, Pengfei Cao, Yuheng Chen, Yubo Chen, Kang Liu, Jun Zhao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对多语言知识编辑中的跨语言更新传播难题，本文构建覆盖12种语言的MKEB基准，并提出MEMLA，通过识别知识神经元并结合神经元掩码的LoRA编辑提升编辑精度、跨语言传播和多跳推理能力。

摘要：

知识编辑旨在调整大语言模型（LLM）中的知识，以防止其回答变得过时或不准确。然而，现有知识编辑研究主要在单一语言中开展，这对于多语言语言模型而言并不充分。本文聚焦多语言知识编辑（MKE），该任务要求在多种语言之间传播更新，这一必要性给任务带来了重大挑战。此外，适用于MKE的综合数据集有限，进一步加剧了挑战，阻碍了该领域的进展。因此，我们推出多语言知识编辑基准（MKEB），这是一个包含12种语言并提供完整评估框架的新颖数据集。此外，我们提出一种通过神经元掩码低秩适配增强多语言知识编辑的方法（MEMLA）。具体而言，我们识别出两类知识神经元以提高编辑精度。而且，我们使用神经元掩码执行基于LoRA的编辑，以高效修改参数并促进更新在多种语言间传播。实验表明，我们的方法优于现有基线，并显著增强被编辑模型的多跳推理能力，同时对其下游任务性能影响极小。数据集和代码将公开发布。

英文摘要：

Knowledge editing aims to adjust the knowledge within large language models (LLMs) to prevent their responses from becoming obsolete or inaccurate. However, existing works on knowledge editing are primarily conducted in a single language, which is inadequate for multilingual language models. In this paper, we focus on multilingual knowledge editing (MKE), which requires propagating updates across multiple languages. This necessity poses a significant challenge for the task. Furthermore, the limited availability of a comprehensive dataset for MKE exacerbates this challenge, hindering progress in this area. Hence, we introduce the Multilingual Knowledge Editing Benchmark (MKEB), a novel dataset comprising 12 languages and providing a complete evaluation framework. Additionally, we propose a method that enhances Multilingual knowledge Editing with neuron-Masked Low-Rank Adaptation (MEMLA). Specifically, we identify two categories of knowledge neurons to improve editing precision. Moreover, we perform LoRA-based editing with neuron masks to efficiently modify parameters and facilitate the propagation of updates across multiple languages. Experiments demonstrate that our method outperforms existing baselines and significantly enhances the multi-hop reasoning capability of the edited model, with minimal impact on its downstream task performance. The dataset and code will be made publicly available.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。