arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.02596cs.CL
收藏

面向低资源语言机器翻译的多示例上下文学习的实证研究

An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages

Mila – Quebec AI Institute(魁北克人工智能研究所);McGill University(麦吉尔大学);Athabasca University(阿特拉斯大学);Peking University(北京大学);Howard University(霍华德大学)

Yinhan Lu, Gaganpreet Jhajj, Chen Zhang, Anietie Andy, David Ifeoluwa Adelani

AI总结:

本研究实证分析多示例上下文学习在低资源语言机器翻译中的效果,发现BM25检索可大幅提升数据效率,且ICL能在微调基础上带来额外增益。

摘要:

上下文学习(ICL)使大型语言模型(LLM)能够从少量示例中适应新任务,因此对于预训练中代表性不足的语言具有前景。最近关于多示例ICL的研究表明,现代LLM可以进一步受益于其长上下文窗口所支持的更大ICL示例。然而,这种增益依赖于仔细的示例选择,并且推理成本对于低资源语言社区来说可能过高。在本文中,我们对从英语到十种真正低资源语言(最近添加到FLORES+中)的机器翻译进行了多示例ICL的实证研究。我们分析了检索更多信息性示例、使用域外数据以及按长度排序示例的效果。我们的发现表明,随着示例数量的增加,多示例ICL变得更加有效。更重要的是,我们表明基于BM25的检索显著提高了数据效率:50个检索到的示例大致匹配250个多示例示例,而250个检索到的示例的性能与1000个多示例示例相似。我们进一步表明,ICL在微调的基础上提供了额外的增益。

英文摘要:

In-context learning (ICL) allows large language models (LLMs) to adapt to new tasks from a few examples, making it promising for languages underrepresented in pre-training. Recent work on many-shot ICL suggests that modern LLMs can further benefit from larger ICL examples enabled by their long context windows. However, such gains depend on careful example selection, and the inference cost can be prohibitive for low-resource language communities. In this paper, we present an empirical study of many-shot ICL for machine translation from English into ten truly low-resource languages recently added to FLORES+. We analyze the effects of retrieving more informative examples, using out-of-domain data, and ordering examples by length. Our findings show that many-shot ICL becomes more effective as the number of examples increases. More importantly, we show that BM25-based retrieval substantially improves data efficiency: 50 retrieved examples roughly match 250 many-shot examples, while 250 retrieved examples perform similarly to 1,000 many-shot examples. We further show that ICL provides additional gains on top of fine-tuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑