面向低资源语言的跨语言检索增强提示
Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages
AI总结：
本文提出 PARC 跨语言检索增强提示方法，通过从高资源语言检索语义相似句作为提示，在 10 种低资源语言的三个任务上显著提升零样本性能，并分析了语言相似性与预训练数据量的影响。
摘要：
多语言预训练语言模型（MPLMs）在近期的跨语言迁移实证研究中展现了强大的多语言能力。本文提出了跨语言检索增强提示（Prompts Augmented by Retrieval Crosslingually，PARC）流程，通过从高资源语言（HRL）中检索语义相似的句子作为提示来增强上下文，从而提升低资源语言（LRLs）上的零样本性能。PARC 在三个下游任务（二分类情感分类、主题分类和自然语言推理）上提升了零样本性能，实验使用覆盖 6 个语系、10 种低资源语言的多语言平行测试集，在无标注设置下提升 5.1%，在有标注设置下提升 16.3%。PARC-labeled 还比微调基线高出 3.7%。我们发现，跨语言迁移性能一方面与高资源语言和低资源语言之间的相似性存在显著正相关，另一方面与低资源预训练数据量也存在显著正相关。鲁棒性分析表明，PARC 有潜力在更强大的多语言预训练语言模型上取得更强的性能。
英文摘要：
Multilingual Pretrained Language Models (MPLMs) have shown their strong multilinguality in recent empirical cross-lingual transfer studies. In this paper, we propose the Prompts Augmented by Retrieval Crosslingually (PARC) pipeline to improve the zero-shot performance on low-resource languages (LRLs) by augmenting the context with semantically similar sentences retrieved from a high-resource language (HRL) as prompts. PARC improves the zero-shot performance on three downstream tasks (binary sentiment classification, topic categorization and natural language inference) with multilingual parallel test sets across 10 LRLs covering 6 language families in both unlabeled settings (+5.1%) and labeled settings (+16.3%). PARC-labeled also outperforms the finetuning baseline by 3.7%. We find a significant positive correlation between cross-lingual transfer performance on one side, and the similarity between the high- and low-resource languages as well as the amount of low-resource pretraining data on the other side. A robustness analysis suggests that PARC has the potential to achieve even stronger performance with more powerful MPLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
