CLM-Bench: 评估和分析LLMs在知识编辑中的跨语言不对齐
CLM-Bench: Benchmarking and Analyzing Cross-lingual Misalignment of LLMs in Knowledge Editing
AI总结：
CLM-Bench通过构建文化意识的基准，揭示了LLMs在知识编辑中的跨语言不对齐问题，挑战了现有跨语言转移方法的有效性。
摘要：
知识编辑（KE）已作为一种有前景的范式，用于在不重新训练的情况下更新大型语言模型（LLMs）中的事实。然而，多语言知识编辑（MKE）的进展目前受到有偏见的评估框架的阻碍。我们观察到，现有的MKE基准通常通过机械地将以英语为中心的数据集翻译成目标语言（例如，英语到中文）来构建。这种方法引入了翻译伪影，并忽视了目标语言中特有的文化实体，未能反映LLMs的真实知识分布。为了解决这一问题，我们提出了CLM-Bench，一种文化意识的基准，采用以中文为主的 methodology。我们整理了1,010对高质量的CounterFact对，这些对根植于中文文化背景，并与英文对应项对齐。利用CLM-Bench，我们对代表性的LLMs（例如Llama-3、Qwen2）进行了广泛的实验，并揭示了显著的跨语言不对齐：一种语言中的编辑独立运作，无法传播到另一种语言。我们进一步通过逐层表示分析提供了几何解释，证明中文和英文的编辑向量几乎正交——位于不同的子空间中，而混合语言编辑则表现出这些向量的线性可加性。我们的发现挑战了当前方法在跨语言转移中的有效性，并强调了本土文化基准的重要性。
英文摘要：
Knowledge Editing (KE) has emerged as a promising paradigm for updating facts in Large Language Models (LLMs) without retraining. However, progress in Multilingual Knowledge Editing (MKE) is currently hindered by biased evaluation frameworks. We observe that existing MKE benchmarks are typically constructed by mechanically translating English-centric datasets into target languages (e.g., English-to-Chinese). This approach introduces translation artifacts and neglects culturally specific entities native to the target language, failing to reflect the true knowledge distribution of LLMs. To address this, we propose CLM-Bench, a culture-aware benchmark constructed using a native Chinese-first methodology. We curate 1,010 high-quality CounterFact pairs rooted in Chinese cultural contexts and align them with English counterparts. Using CLM-Bench, we conduct extensive experiments on representative LLMs (e.g., Llama-3, Qwen2) and reveal a significant Cross-lingual Misalignment: edits in one language function independently and fail to propagate to the other. We further provide a geometric explanation via layer-wise representation analysis, demonstrating that edit vectors for Chinese and English are nearly orthogonal -- residing in disjoint subspaces -- while mixed-lingual editing exhibits linear additivity of these vectors. Our findings challenge the effectiveness of current methods in cross-lingual transfer and underscore the importance of culturally native benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
