arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.01679cs.CLcs.AI
收藏

基于预训练语言模型的自适应少样本提示机器翻译

Adaptive Few-shot Prompting for Machine Translation with Pre-trained Language Models

Guangdong University of Technology(广东工业大学);The Chinese University of Hong Kong(香港中文大学)

Lei Tang, Jinghui Qin, Wenxuan Ye, Hao Tan, Zhijing Yang

AI总结:

本文提出自适应少样本提示框架AFSP,通过基于LLM嵌入的示例检索和候选重排序提升机器翻译效果,并构建外交中英平行数据集验证其有效性。

摘要:

近年来,具备上下文学习能力的大型语言模型(LLMs)在处理神经机器翻译方面展现出显著潜力。然而,现有证据表明,大型语言模型对提示敏感,在下游机器翻译任务中对任意输入使用固定提示并非最优。为解决这一问题,我们提出了一种自适应少样本提示(AFSP)框架,能够为不同的源输入句子自动选择合适的翻译示例,从而进一步激发大型语言模型的翻译能力,以实现更好的机器翻译。首先,我们基于大型语言模型的嵌入构建了一个翻译示例检索模块,从对齐的平行翻译语料库中检索语义最相似的 top-k 个翻译示例。我们没有使用其他嵌入模型进行语义示例检索,而是基于所部署大型语言模型的嵌入层构建了一个混合示例检索模块,以构建更好的输入表示,从而检索出语义更相关的翻译示例。然后,为了确保源输入与目标输出之间具有更好的语义一致性,我们借助翻译示例强制所部署的大型语言模型自身生成多个目标语言候选输出,并对这些候选输出进行重排序。此外,为了更好地评估我们的 AFSP 框架在最新语言上的有效性,并拓展神经机器翻译的研究边界,我们构建了一个高质量的外交领域中文-英文平行数据集,该数据集包含 5,528 对中英平行句子。最后,在所提出的外交中英平行数据集和联合国平行语料库(中英部分)上进行的大量实验表明,我们提出的 AFSP 框架具有有效性和优越性。

英文摘要:

Recently, Large language models (LLMs) with in-context learning have demonstrated remarkable potential in handling neural machine translation. However, existing evidence shows that LLMs are prompt-sensitive and it is sub-optimal to apply the fixed prompt to any input for downstream machine translation tasks. To address this issue, we propose an adaptive few-shot prompting (AFSP) framework to automatically select suitable translation demonstrations for various source input sentences to further elicit the translation capability of an LLM for better machine translation. First, we build a translation demonstration retrieval module based on LLM's embedding to retrieve top-k semantic-similar translation demonstrations from aligned parallel translation corpus. Rather than using other embedding models for semantic demonstration retrieval, we build a hybrid demonstration retrieval module based on the embedding layer of the deployed LLM to build better input representation for retrieving more semantic-related translation demonstrations. Then, to ensure better semantic consistency between source inputs and target outputs, we force the deployed LLM itself to generate multiple output candidates in the target language with the help of translation demonstrations and rerank these candidates. Besides, to better evaluate the effectiveness of our AFSP framework on the latest language and extend the research boundary of neural machine translation, we construct a high-quality diplomatic Chinese-English parallel dataset that consists of 5,528 parallel Chinese-English sentences. Finally, extensive experiments on the proposed diplomatic Chinese-English parallel dataset and the United Nations Parallel Corpus (Chinese-English part) show the effectiveness and superiority of our proposed AFSP.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑