arXiv 2311.005872023-12-02cs.CL

面向孟加拉语的跨语言检索增强上下文学习

Crosslingual Retrieval Augmented In-context Learning for Bangla

Center for Information and Language Processing (CIS), LMU Munich（慕尼黑大学信息与语言处理中心（CIS））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML））

Xiaoqian Li, Ercong Nie, Sheng Liang

备注：: In The 1st Bangla Language Processing (BLP) Workshop, held in conjunction with The Conference on Empirical Methods in Natural Language Processing (EMNLP), December 2023

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出跨语言检索增强上下文学习方法，通过从高资源语言检索语义相似提示来增强BLOOMZ等多语言预训练模型，从而稳定提升其在孟加拉语低资源任务上的零样本表现。

摘要：

大型语言模型（LLMs）在自然语言处理中的前景常常因其在孟加拉语等低资源语言上的有限表现而蒙上阴影。为了解决这一问题，本文提出了一种利用跨语言检索增强上下文学习的开创性方法。通过策略性地从高资源语言中检索语义相似的提示，我们使多语言预训练语言模型（MPLMs），尤其是生成式模型BLOOMZ，能够成功提升在孟加拉语任务上的性能。我们广泛的评估表明，跨语言检索增强提示在零样本（zero-shot）性能基础上为MPLMs带来了稳定的改进。

英文摘要：

The promise of Large Language Models (LLMs) in Natural Language Processing has often been overshadowed by their limited performance in low-resource languages such as Bangla. To address this, our paper presents a pioneering approach that utilizes cross-lingual retrieval augmented in-context learning. By strategically sourcing semantically similar prompts from high-resource language, we enable multilingual pretrained language models (MPLMs), especially the generative model BLOOMZ, to successfully boost performance on Bangla tasks. Our extensive evaluation highlights that the cross-lingual retrieval augmented prompts bring steady improvements to MPLMs over the zero-shot performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。