通过提示学习缓解开放域对话生成跨语言迁移中的灾难性遗忘
Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation
AI总结：
本文针对非英语开放域对话生成中少样本跨语言迁移与多任务学习的灾难性遗忘问题，提出基于固定提示语言模型调优和手工提示的提示学习方法，在6种语言上验证了其有效性。
摘要：
非英语语言的对话系统长期以来未得到充分探索。本文首次在数据有限的非英语语言开放域对话生成背景下，研究少样本跨语言迁移学习（FS-XLT）与多任务学习（MTL）。在初步实验中，我们观察到全部6种语言在FS-XLT和MTL中均出现灾难性遗忘。为缓解该问题，我们提出一种简单而有效的提示学习方法，通过固定提示语言模型调优（Fixed-prompt LM Tuning）与手工设计的提示，弥合预训练与微调之间的差距，从而在多语言预训练语言模型（mPLM）中保持FS-XLT和MTL的多语言能力。在全部6种语言上的自动评估与人工评估结果均证明了该方法的有效性。代码见 https://github.com/JeremyLeiLiu/XLinguDial。
英文摘要：
Dialogue systems for non-English languages have long been under-explored. In this paper, we take the first step to investigate few-shot cross-lingual transfer learning (FS-XLT) and multitask learning (MTL) in the context of open-domain dialogue generation for non-English languages with limited data. We observed catastrophic forgetting in both FS-XLT and MTL for all 6 languages in our preliminary experiments. To mitigate the issue, we propose a simple yet effective prompt learning approach that can preserve the multilinguality of multilingual pre-trained language model (mPLM) in FS-XLT and MTL by bridging the gap between pre-training and fine-tuning with Fixed-prompt LM Tuning and our hand-crafted prompts. Experimental results on all 6 languages in terms of both automatic and human evaluations demonstrate the effectiveness of our approach. Our code is available at https://github.com/JeremyLeiLiu/XLinguDial.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
