一次编辑,处处更新:大语言模型中跨语言知识同步的简单框架
Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs
AI总结:
提出X-KDE框架,通过两阶段方法(XE-IT和TL-PO)实现大语言模型跨语言知识同步,在Bi-ZsRE和MzsRE基准上平均提升+8.19%,同时保持单语言高精度。
摘要:
知识编辑允许对大型语言模型(LLMs)进行高效调整,以适应新信息或修正,而无需完全重新训练。然而,先前的方法通常侧重于单语言编辑或基础的多语言编辑,未能实现真正的跨语言知识同步。为解决这一问题,我们提出了一种简单实用的最先进(SOTA)方案——跨语言知识民主编辑(X-KDE),旨在将知识从主导语言有效地传播到其他语言。我们的X-KDE包含两个阶段:(i)跨语言编辑指令微调(XE-IT),在精选的平行数据集上对模型进行微调,以修改范围内的知识,同时保留无关信息;(ii)目标语言偏好优化(TL-PO),应用先进的优化技术确保跨语言一致性,促进更新的迁移。此外,我们贡献了一个高质量的跨语言数据集,专门设计用于增强跨语言的知识迁移。在Bi-ZsRE和MzsRE基准上的广泛实验表明,X-KDE显著提升了跨语言性能,平均提高了+8.19%,同时在单语言设置中保持了高准确性。
英文摘要:
Knowledge editing allows for efficient adaptation of large language models (LLMs) to new information or corrections without requiring full retraining. However, prior methods typically focus on either single-language editing or basic multilingual editing, failing to achieve true cross-linguistic knowledge synchronization. To address this, we present a simple and practical state-of-the-art (SOTA) recipe Cross-Lingual Knowledge Democracy Edit (X-KDE), designed to propagate knowledge from a dominant language to other languages effectively. Our X-KDE comprises two stages: (i) Cross-lingual Edition Instruction Tuning (XE-IT), which fine-tunes the model on a curated parallel dataset to modify in-scope knowledge while preserving unrelated information, and (ii) Target-language Preference Optimization (TL-PO), which applies advanced optimization techniques to ensure consistency across languages, fostering the transfer of updates. Additionally, we contribute a high-quality, cross-lingual dataset, specifically designed to enhance knowledge transfer across languages. Extensive experiments on the Bi-ZsRE and MzsRE benchmarks show that X-KDE significantly enhances cross-lingual performance, achieving an average improvement of +8.19%, while maintaining high accuracy in monolingual settings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。