arXiv 2411.163002024-12-19cs.CLcs.AI

BayLing 2：一种具有高效语言对齐的多语言大语言模型

BayLing 2: A Multilingual Large Language Model with Efficient Language Alignment

Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Key Laboratory of Intelligent Information Processing（智能信息处理重点实验室）；Research Center of Distributed Systems（分布式系统研究中心）；Key Laboratory of AI Safety（人工智能安全重点实验室）

Shaolei Zhang, Kehao Zhang, Qingkai Fang, Shoutao Guo, Yan Zhou, Xiaodong Liu, Yang Feng

备注：: BayLing 2's online demo: http://nlp.ict.ac.cn/bayling/demo. BayLing 2's code and models: https://github.com/ictnlp/BayLing

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出多语言大模型BayLing 2，通过构建320万指令数据集进行语言对齐，将高资源语言能力高效转移至100多种低资源语言，在翻译与知识理解任务上取得显著提升。

摘要：

大语言模型（LLMs）凭借其强大的生成能力和海量知识，为日常生活中的各种任务赋能。然而，这些能力主要集中在高资源语言上，使得低资源语言的生成能力较弱且知识相对有限。因此，增强LLMs的多语言能力对于服务全球100多种语言社区至关重要。增强多语言能力的一种直观方法是为各种语言构建指令数据，但为100多种语言构建指令数据的成本过高。在本文中，我们介绍了BayLing 2，它通过语言对齐将生成能力和知识从高资源语言高效地转移到低资源语言。为此，我们构建了一个包含320万条指令的数据集，该数据集包含高资源语言指令（中文和英文）以及100多种语言的跨语言指令，并基于该数据集进行指令微调以促进语言之间的能力转移。我们使用Llama作为基础模型，开发了BayLing-2-7B、BayLing-2-13B和BayLing-2-8B，并对BayLing进行了全面评估。在100多种语言的多语言翻译任务中，BayLing与同等规模的开源模型相比表现出优越的性能。在多语言知识和理解基准测试中，BayLing在20多种低资源语言上取得了显著提升，证明了其将知识从高资源语言有效转移到低资源语言的能力。此外，英文基准测试结果表明，BayLing在增强低资源语言性能的同时，保持了在高资源语言上的高性能。BayLing的Demo、主页、代码和模型均已公开。

英文摘要：

Large language models (LLMs), with their powerful generative capabilities and vast knowledge, empower various tasks in everyday life. However, these abilities are primarily concentrated in high-resource languages, leaving low-resource languages with weaker generative capabilities and relatively limited knowledge. Enhancing the multilingual capabilities of LLMs is therefore crucial for serving over 100 linguistic communities worldwide. An intuitive approach to enhance the multilingual capabilities would be to construct instruction data for various languages, but constructing instruction data for over 100 languages is prohibitively costly. In this paper, we introduce BayLing 2, which efficiently transfers generative capabilities and knowledge from high-resource languages to low-resource languages through language alignment. To achieve this, we constructed a dataset of 3.2 million instructions, comprising high-resource language instructions (Chinese and English) and cross-lingual instructions for 100+ languages and performed instruction tuning based on the dataset to facilitate the capability transfer between languages. Using Llama as the foundation model, we developed BayLing-2-7B, BayLing-2-13B, and BayLing-2-8B, and conducted a comprehensive evaluation of BayLing. For multilingual translation across 100+ languages, BayLing shows superior performance compared to open-source models of similar scale. For multilingual knowledge and understanding benchmarks, BayLing achieves significant improvements across over 20 low-resource languages, demonstrating its capability of effective knowledge transfer from high-resource to low-resource languages. Furthermore, results on English benchmarks indicate that BayLing maintains high performance in highresource languages while enhancing the performance in low-resource languages. Demo, homepage, code and models of BayLing are available.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。