arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.07255cs.CLcs.AI
收藏

基于双重提示增强的跨语言提示增强

Enhancing Cross-lingual Prompting with Dual Prompt Augmentation

Carnegie Mellon University(卡内基梅隆大学);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院)

Meng Zhou, Xin Li, Yue Jiang, Lidong Bing

AI总结:

本文提出双重提示增强框架DPA,通过语言无关的Universal Prompting和双重提示增强缓解跨语言提示中的数据稀缺问题,在XNLI任务上以16个英语样本/类达到46.54%,显著优于微调的34.99%。

摘要:

提示在少样本场景中展现出令人鼓舞的结果。然而,其对于多语言/跨语言问题的潜力尚未被充分挖掘。Zhao和Schütze(2021)通过展示跨语言提示优于跨语言微调,在该方向进行了初步探索。本文对跨语言提示中各组件的效果进行了实证探索,并推导出语言无关的Universal Prompting,有助于缓解源语言训练与目标语言推理之间的差异。在此基础上,我们提出DPA,一个双重提示增强框架,旨在缓解少样本跨语言提示中的数据稀缺问题。值得注意的是,对于XNLI,我们的方法在每类仅有16个英语训练样本的情况下达到了46.54%,显著优于微调的34.99%。我们的代码可在https://github.com/DAMO-NLP-SG/DPA获取。

英文摘要:

Prompting shows promising results in few-shot scenarios. However, its strength for multilingual/cross-lingual problems has not been fully exploited. Zhao and Schütze (2021) made initial explorations in this direction by presenting that cross-lingual prompting outperforms cross-lingual finetuning. In this paper, we conduct an empirical exploration on the effect of each component in cross-lingual prompting and derive language-agnostic Universal Prompting, which helps alleviate the discrepancies between source-language training and target-language inference. Based on this, we propose DPA, a dual prompt augmentation framework, aiming at relieving the data scarcity issue in few-shot cross-lingual prompting. Notably, for XNLI, our method achieves 46.54% with only 16 English training examples per class, significantly better than 34.99% of finetuning. Our code is available at https://github.com/DAMO-NLP-SG/DPA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑