多语言语言模型中事实知识的跨语言一致性
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
AI总结：
该研究针对多语言预训练语言模型事实知识的跨语言一致性问题，提出RankC评估指标，分析其影响因素，并通过模型编辑案例发现新知识仅向与英语RankC得分高的语言迁移。
摘要：
多语言大规模预训练语言模型（PLMs）已被证明存储了大量事实知识，但不同语言之间存在显著差异。我们以确保不同语言背景的用户从同一模型获得一致反馈为最终目标，研究了多种多语言PLM中事实知识的跨语言一致性（CLC）。为此，我们提出了一种基于排序的一致性（RankC）指标，用于独立于准确率评估不同语言间的知识一致性。利用该指标，我们从模型层面和语言对层面对CLC的决定因素进行了深入分析。在诸多结果中，我们发现增大模型规模会提升大多数语言的事实探测准确率，但无法改善跨语言一致性。最后，我们针对通过模型编辑向PLM中插入新事实关联时的CLC开展了案例研究。对以英语插入的小样本事实的研究结果揭示了一个清晰模式：新知识仅会迁移到与英语RankC得分较高的语言。
英文摘要：
Multilingual large-scale Pretrained Language Models (PLMs) have been shown to store considerable amounts of factual knowledge, but large variations are observed across languages. With the ultimate goal of ensuring that users with different language backgrounds obtain consistent feedback from the same model, we study the cross-lingual consistency (CLC) of factual knowledge in various multilingual PLMs. To this end, we propose a Ranking-based Consistency (RankC) metric to evaluate knowledge consistency across languages independently from accuracy. Using this metric, we conduct an in-depth analysis of the determining factors for CLC, both at model level and at language-pair level. Among other results, we find that increasing model size leads to higher factual probing accuracy in most languages, but does not improve cross-lingual consistency. Finally, we conduct a case study on CLC when new factual associations are inserted in the PLMs via model editing. Results on a small sample of facts inserted in English reveal a clear pattern whereby the new piece of knowledge transfers only to languages with which English has a high RankC score.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
