MLaKE：面向大型语言模型的多语言知识编辑基准
MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models
AI总结：
本文提出 MLaKE 多语言知识编辑基准，覆盖五种语言及单跳、多跳问答，用于评估现有方法的跨语言泛化能力并揭示其局限。
摘要：
大型语言模型（LLM）的广泛应用凸显了在其内在参数中嵌入准确且与时俱进的知识的关键必要性。现有知识编辑研究主要集中于单语言场景，忽视了多语言语境和多跳推理所带来的复杂性。为应对这些挑战，本研究提出 MLaKE（Multilingual Language Knowledge Editing，多语言语言知识编辑），这是一个包含4072个多跳问题和5360个单跳问题的新型基准，旨在评估知识编辑方法在英语、中文、日语、法语和德语五种语言中的适应性。MLaKE 聚合来自跨语言 Wikipedia 的事实链，并利用 LLM 生成自由形式和多项选择两种形式的问题。我们在 MLaKE 上评估了现有方法的多语言知识编辑泛化能力。现有知识编辑方法在英语样本上表现出比其他语言更高的成功率。然而，在多语言实验中，它们的泛化能力有限。值得注意的是，与不同语系的语言相比，现有知识编辑方法通常对同一语系内的语言表现出相对较高的泛化能力。这些结果强调了推进多语言知识编辑的迫切需求，我们希望 MLaKE 能够成为基准测试和解决方案开发的宝贵资源。
英文摘要：
The extensive utilization of large language models (LLMs) underscores the crucial necessity for precise and contemporary knowledge embedded within their intrinsic parameters. Existing research on knowledge editing primarily concentrates on monolingual scenarios, neglecting the complexities presented by multilingual contexts and multi-hop reasoning. To address these challenges, our study introduces MLaKE (Multilingual Language Knowledge Editing), a novel benchmark comprising 4072 multi-hop and 5360 single-hop questions designed to evaluate the adaptability of knowledge editing methods across five languages: English, Chinese, Japanese, French, and German. MLaKE aggregates fact chains from Wikipedia across languages and utilizes LLMs to generate questions in both free-form and multiple-choice. We evaluate the multilingual knowledge editing generalization capabilities of existing methods on MLaKE. Existing knowledge editing methods demonstrate higher success rates in English samples compared to other languages. However, their generalization capabilities are limited in multi-language experiments. Notably, existing knowledge editing methods often show relatively high generalization for languages within the same language family compared to languages from different language families. These results underscore the imperative need for advancements in multilingual knowledge editing and we hope MLaKE can serve as a valuable resource for benchmarking and solution development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
