弥合语言鸿沟：预训练和数据集中的跨语言映射以提升多语言大语言模型性能
Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance
AI总结：
本文提出在预训练阶段引入跨语言映射任务，提升多语言对齐能力，通过语言对齐系数量化一致性，实验显示在机器翻译、跨语言自然语言理解及问答任务中性能显著提升。
摘要：
多语言大语言模型（LLMs）在跨语言任务中面临高资源语言与低资源语言数据不平衡以及预训练中的单语偏见问题。现有方法如双语微调和对比对齐虽能提升跨语言性能，但常需大量平行数据或存在不稳定问题。为此，我们引入预训练阶段的跨语言映射任务，增强跨语言对齐而不影响单语流畅性。我们的方法在LLM嵌入空间内双向映射语言，提升语言生成与理解能力。进一步提出语言对齐系数以稳健量化跨语言一致性，即使在数据有限场景下也能有效工作。实验结果表明，在机器翻译（MT）、跨语言自然语言理解（CLNLU）和跨语言问答（CLQA）任务中，我们的模型在MT中达到最高11.9 BLEU点提升，在CLQA BERTScore-Precision中提升6.72点，在CLNLU准确率上超过强多语言基线5%以上。这些发现凸显了在预训练中整合跨语言目标以提升多语言LLM性能的潜力。
英文摘要：
Multilingual Large Language Models (LLMs) struggle with cross-lingual tasks due to data imbalances between high-resource and low-resource languages, as well as monolingual bias in pre-training. Existing methods, such as bilingual fine-tuning and contrastive alignment, can improve cross-lingual performance, but they often require extensive parallel data or suffer from instability. To address these challenges, we introduce a Cross-Lingual Mapping Task during the pre-training phase, which enhances cross-lingual alignment without compromising monolingual fluency. Our approach bi-directionally maps languages within the LLM embedding space, improving both language generation and comprehension. We further propose a Language Alignment Coefficient to robustly quantify cross-lingual consistency, even in limited-data scenarios. Experimental results on machine translation (MT), cross-lingual natural language understanding (CLNLU), and cross-lingual question answering (CLQA) show that our model achieves gains of up to 11.9 BLEU points in MT, 6.72 points in CLQA BERTScore-Precision, and more than 5% in CLNLU accuracy over strong multilingual baselines. These findings highlight the potential of incorporating cross-lingual objectives into pre-training to improve multilingual LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
