BMIKE-53:基于上下文学习的跨语言知识编辑研究
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
AI总结:
本文提出跨语言上下文知识编辑基准BMIKE-53,统一三个数据集并在多语言下系统评估IKE,发现模型规模、示例对齐与语言字符类型是影响性能的关键因素。
摘要:
本文提出了BMIKE-53,这是一个涵盖53种语言的跨语言上下文知识编辑(IKE)综合基准,统一了zsRE、CounterFact和WikiFactDiff三个知识编辑(KE)数据集。跨语言KE要求在一种语言中编辑的知识能够泛化到其他语言,同时保留无关知识,目前仍研究不足。为填补这一空白,我们在零样本、单样本和少样本设置下系统评估了IKE,并引入了量身定制的特定指标示例。研究结果表明,模型规模和示例对齐对跨语言IKE效果起关键决定作用,更大的模型和定制示例能显著提升性能。语言特性(尤其是字符类型)强烈影响不同语言间的性能差异,非拉丁语系语言因语言混淆等问题表现不佳。代码和数据已公开于:https://github.com/ercong21/MultiKnow/。
英文摘要:
This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE) across 53 languages, unifying three knowledge editing (KE) datasets: zsRE, CounterFact, and WikiFactDiff. Cross-lingual KE, which requires knowledge edited in one language to generalize across others while preserving unrelated knowledge, remains underexplored. To address this gap, we systematically evaluate IKE under zero-shot, one-shot, and few-shot setups, incorporating tailored metric-specific demonstrations. Our findings reveal that model scale and demonstration alignment critically govern cross-lingual IKE efficacy, with larger models and tailored demonstrations significantly improving performance. Linguistic properties, particularly script type, strongly influence performance variation across languages, with non-Latin languages underperforming due to issues like language confusion. Code and data are publicly available at: https://github.com/ercong21/MultiKnow/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。