利用 Albertina PT* 家族培育葡萄牙语开放神经编码器生态系统
Fostering the Ecosystem of Open Neural Encoders for Portuguese with Albertina PT* Family
AI总结：
本文扩展葡萄牙语开放编码器生态，推出15亿参数高性能模型与1亿参数高效模型，并公开基于 SuperGLUE 的葡萄牙语数据集。
摘要：
为促进葡萄牙语的神经编码，本文贡献了基础编码器模型，这些模型代表了对仍然非常稀缺的、专门为该语言开发的大语言模型生态系统的扩展，并且是完全开放的，即它们是开源的，并在开放许可下为任何目的（包括研究和商业用途）免费公开分发。与英语以外的大多数语言一样，葡萄牙语在这些基础语言资源方面属于低资源语言，现有开创性模型包括9亿参数的 Albertina 和3.35亿参数的 Bertimbau。本文将这两个模型作为初始集合，提出扩展葡萄牙语最先进开放编码器生态系统：一个由顶级性能驱动、拥有15亿参数的较大模型，以及一个由效率驱动、拥有1亿参数的较小模型。在实现这一主要目标的同时，本文还获得了与该生态系统相关的进一步成果，即基于 SuperGLUE 基准的葡萄牙语新数据集，我们也将其公开分发。
英文摘要：
To foster the neural encoding of Portuguese, this paper contributes foundation encoder models that represent an expansion of the still very scarce ecosystem of large language models specifically developed for this language that are fully open, in the sense that they are open source and openly distributed for free under an open license for any purpose, thus including research and commercial usages. Like most languages other than English, Portuguese is low-resourced in terms of these foundational language resources, there being the inaugural 900 million parameter Albertina and 335 million Bertimbau. Taking this couple of models as an inaugural set, we present the extension of the ecosystem of state-of-the-art open encoders for Portuguese with a larger, top performance-driven model with 1.5 billion parameters, and a smaller, efficiency-driven model with 100 million parameters. While achieving this primary goal, further results that are relevant for this ecosystem were obtained as well, namely new datasets for Portuguese based on the SuperGLUE benchmark, which we also distribute openly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
