发表机构
Linköping University（林雪平大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出基于干预的框架，构建SIFT资源，发现多语言模型跨语言事实迁移有限，源语言实体频率影响显著，简单负候选集夸大表观迁移。
AI 中文摘要
多语言语言模型在持续预训练过程中会跨语言迁移事实知识，还是大多会直接回忆从目标语言数据中学习到的事实？为更可靠地回答该问题，我们提出一种基于干预的框架：从预训练的英语模型出发，在不同粒度级别上系统性移除特定事实的波斯语数据上进行持续预训练。我们构建了SIFT这一资源，包含20种关系下的500个三元组，按每个事实主体的文化起源分层为通用（全球知名）和波斯相关实体，用于训练数据中的系统性事实移除与评估，配有原生编写的波斯语完形填空模板。我们的结果表明，事实迁移非常有限：在最严格的移除条件下，绝大多数英语习得的事实无法迁移到波斯语中。我们进一步发现，句子级共现移除不足以消除事实信号，且更简单（随机选择）的负候选集通过奖励浅层联想启发式方法，大幅夸大了表观迁移，而对更难的候选集（减少对启发式方法的依赖）的性能则低得多。最后，我们表明源语言实体频率有很大影响，在英语语料库中稀有几个数量级的波斯相关事实几乎无法迁移。
英文摘要
Do multilingual language models transfer factual knowledge across languages during continued pretraining, or do they mostly recall facts learned directly from the target-language data? To answer this question more reliably, we propose an intervention-based framework: starting from an English-pretrained model, we continue pretraining on Persian data from which specific facts have been systematically removed at varying levels of granularity. We construct SIFT, a resource of 500 triples across 20 relations, stratified by the cultural origin of each fact's subject into general (globally prominent) and Persian-related entities, designed for both systematic fact removal from training data and evaluation, with natively written Persian cloze templates. Our results show that fact transfer is very limited: under the strictest removal condition, a large majority of English-acquired facts fail to transfer into Persian. We further show that sentence-level co-occurrence removal is insufficient to eliminate fact signal, and that easier (randomly selected) negative candidate sets substantially inflate apparent transfer by rewarding shallow associative heuristics, while performance on a harder candidate set that allows for less reliance on heuristics is much lower. Finally, we show that source-language entity frequency has a large influence, with Persian-related facts, which are orders of magnitude rarer in the English corpus, hardly transferring.
CommentsAccepted at EMNLP 2026 Main Conference
