Polyglot-Ko技术报告：开源大规模韩语语言模型
A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean Language Models
AI总结：
针对多语言模型韩语能力不足的问题，本文介绍Polyglot Korean模型，并利用与TUNiB合作整理的1.2TB韩语数据优先训练大规模韩语模型，以支持性能比较及韩国产业与研究需求。
摘要：
Polyglot是一个开创性项目，旨在提升多语言语言模型在非英语语言上的性能。尽管已有mBERT（Devlin等人，2019）、XGLM（Lin等人，2022）和BLOOM（Scao等人，2022）等多种多语言模型可用，但由于对当前多语言模型的非英语语言能力不满意，研究人员和开发者往往不得不使用各自语言构建单语模型。为解决这一差距，我们寻求开发能够提升非英语语言性能的先进多语言语言模型。本文介绍Polyglot Korean模型，它们代表了一个特定重点，而本质上并非多语言模型。通过与TUNiB合作，我们团队为研究工作精心收集并整理了1.2TB韩语数据。我们有意决定在涉足多语言模型之前优先开发韩语模型。这一选择由多个因素推动：首先，韩语模型便于与现有多语言模型进行性能比较；最后，它们能够满足韩国企业和研究人员的特定需求。本文展示了我们开发Polyglot Korean模型的工作，为解决多语言语言模型中的非英语语言性能差距提出了若干步骤。
英文摘要：
Polyglot is a pioneering project aimed at enhancing the non-English language performance of multilingual language models. Despite the availability of various multilingual models such as mBERT (Devlin et al., 2019), XGLM (Lin et al., 2022), and BLOOM (Scao et al., 2022), researchers and developers often resort to building monolingual models in their respective languages due to the dissatisfaction with the current multilingual models non-English language capabilities. Addressing this gap, we seek to develop advanced multilingual language models that offer improved performance in non-English languages. In this paper, we introduce the Polyglot Korean models, which represent a specific focus rather than being multilingual in nature. In collaboration with TUNiB, our team collected 1.2TB of Korean data meticulously curated for our research journey. We made a deliberate decision to prioritize the development of Korean models before venturing into multilingual models. This choice was motivated by multiple factors: firstly, the Korean models facilitated performance comparisons with existing multilingual models; and finally, they catered to the specific needs of Korean companies and researchers. This paper presents our work in developing the Polyglot Korean models, which propose some steps towards addressing the non-English language performance gap in multilingual language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
