LEIA：通过基于实体的数据增强促进语言模型中的跨语言知识迁移
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation
AI总结：
LEIA提出一种利用跨语言对齐的维基百科实体名称增强目标语言语料并进行语言建模训练的方法，在多种非英语问答任务上显著提升了70亿参数大语言模型的跨语言迁移性能。
摘要：
由于跨语言迁移的高效性和潜力，将基于英语的大语言模型（LLMs）适配到其他语言已变得越来越流行。然而，现有的语言适配方法往往忽视了跨语言监督的益处。在本研究中，我们提出了LEIA，一种利用跨语言对齐的维基百科实体名称的语言适配调优方法。该方法通过用英语实体名称增强目标语言语料库，并使用从左到右的语言建模来训练模型。我们在多种问答数据集上使用70亿参数的LLMs评估LEIA，展示了在各种非英语语言上的显著性能提升。源代码可在https://github.com/studio-ousia/leia获取。
英文摘要：
Adapting English-based large language models (LLMs) to other languages has become increasingly popular due to the efficiency and potential of cross-lingual transfer. However, existing language adaptation methods often overlook the benefits of cross-lingual supervision. In this study, we introduce LEIA, a language adaptation tuning method that utilizes Wikipedia entity names aligned across languages. This method involves augmenting the target language corpus with English entity names and training the model using left-to-right language modeling. We assess LEIA on diverse question answering datasets using 7B-parameter LLMs, demonstrating significant performance gains across various non-English languages. The source code is available at https://github.com/studio-ousia/leia.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
