语言各向异性跨语言模型编辑
Language Anisotropic Cross-Lingual Model Editing
AI总结：
本文定义跨语言模型编辑任务并提出基于平行语料的适配框架和语言各向异性编辑方法，通过放大不同语言对应的参数子集提升编辑跨语言传播效果，实验验证了其有效性。
摘要：
多语言预训练语言模型能够学习任务特定能力或记忆跨多种语言的事实，但在特定输入上不可避免地会产生不期望的预测。基于类似的观察，模型编辑旨在事后针对特定输入校准模型，同时保持模型的原始行为。然而，现有工作仅研究单语场景，缺乏跨语言迁移能力，无法同时跨语言执行编辑。在本工作中，我们聚焦于跨语言模型编辑。首先，我们定义了跨语言模型编辑任务及相应指标，其中一种语言的编辑会传播到其他语言。其次，我们提出了一个框架，利用平行语料将单语模型编辑方法自然适配到跨语言场景。此外，我们提出了语言各向异性编辑，通过为每种语言放大不同的参数子集来改进跨语言编辑。在新定义的跨语言模型编辑任务上，我们通过实验证明了单语基线在将编辑传播到多种语言方面的失败，以及所提出的语言各向异性模型编辑的有效性。我们的代码已在 https://github.com/franklear/LiME 公开。
英文摘要：
Multilingual pre-trained language models can learn task-specific abilities or memorize facts across multiple languages but inevitably make undesired predictions with specific inputs. Under similar observation, model editing aims to post-hoc calibrate a model targeted to specific inputs with keeping the model's raw behavior. However, existing work only studies the monolingual scenario, which lacks the cross-lingual transferability to perform editing simultaneously across languages. In this work, we focus on cross-lingual model editing. Firstly, we define the cross-lingual model editing task and corresponding metrics, where an edit in one language propagates to the others. Next, we propose a framework to naturally adapt monolingual model editing approaches to the cross-lingual scenario using parallel corpus. Further, we propose language anisotropic editing to improve cross-lingual editing by amplifying different subsets of parameters for each language. On the newly defined cross-lingual model editing task, we empirically demonstrate the failure of monolingual baselines in propagating the edit to multiple languages and the effectiveness of the proposed language anisotropic model editing. Our code is publicly available at https://github.com/franklear/LiME.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
