arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.17652cs.CL
收藏

Qomhra: 一种双语爱尔兰语和英语大语言模型

Qomhra: A Bilingual Irish and English Large Language Model

Trinity College Dublin(三一学院);University College Cork(科克大学);Queen’s University Belfast(贝尔法斯特女王大学)

Joseph McInerney, Khanh-Tung Tran, Liam Lonergan, Ailbhe Ní Chasaide, Neasa Ní Chiaráin, Barry Devereux

AI总结:

Qomhra是一种双语爱尔兰语和英语大语言模型,通过低资源约束下的方法生成人类偏好数据,显著提升了爱尔兰语和英语的性能表现。

摘要:

大型语言模型(LLM)的研究和开发一直聚焦于世界主要语言,导致低资源语言如爱尔兰语的代表性不足。本文介绍了Qomhrá,一种双语爱尔兰语和英语LLM,其开发在极低资源约束下完成。文中概述了一个完整的流程,涵盖双语持续预训练、指令微调以及为未来对齐训练合成的人类偏好数据。我们关注缺乏可扩展方法来创建人类偏好数据的问题,通过提出一种新的方法来生成此类数据,即通过提示LLM生成“接受”和“拒绝”的响应,我们验证其与L1爱尔兰语使用者一致。为了选择用于合成的LLM,我们评估了顶级闭源LLM在爱尔兰语生成性能上的表现。Gemini-2.5-Pro在L1和L2爱尔兰语使用者中排名最高,与LLM作为评判者评分不同,表明当前LLM与爱尔兰语社区之间存在不一致。随后,我们利用Gemini-2.5-Pro将大规模英语指令微调数据集翻译成爱尔兰语,并合成首个爱尔兰语人类偏好数据集。我们全面评估Qomhrá在多个基准上的表现,测试翻译、性别理解、主题识别和世界知识;这些评估显示,与现有开源爱尔兰语LLM基线UCCIX相比,爱尔兰语提升了最高29%,英语提升了44%。我们的框架结果为开发爱尔兰语和其他低资源语言的LLM提供了洞察和指导。

英文摘要:

Large language model (LLM) research and development has overwhelmingly focused on the world's major languages, leading to under-representation of low-resource languages such as Irish. This paper introduces \textbf{Qomhrá}, a bilingual Irish and English LLM, developed under extremely low-resource constraints. A complete pipeline is outlined spanning bilingual continued pre-training, instruction tuning, and the synthesis of human preference data for future alignment training. We focus on the lack of scalable methods to create human preference data by proposing a novel method to synthesise such data by prompting an LLM to generate ``accepted'' and ``rejected'' responses, which we validate as aligning with L1 Irish speakers. To select an LLM for synthesis, we evaluate the top closed-weight LLMs for Irish language generation performance. Gemini-2.5-Pro is ranked highest by L1 and L2 Irish-speakers, diverging from LLM-as-a-judge ratings, indicating a misalignment between current LLMs and the Irish-language community. Subsequently, we leverage Gemini-2.5-Pro to translate a large scale English-language instruction tuning dataset to Irish and to synthesise a first-of-its-kind Irish-language human preference dataset. We comprehensively evaluate Qomhrá across several benchmarks, testing translation, gender understanding, topic identification, and world knowledge; these evaluations show gains of up to 29\% in Irish and 44\% in English compared to the existing open-source Irish LLM baseline, UCCIX. The results of our framework provide insight and guidance to developing LLMs for both Irish and other low-resource languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑