基于提示调优的对话任务高效对齐跨语言迁移学习
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning
AI总结：
该研究构建了含105种语言、单语约33万话语的跨语言对话数据集XSGD，提出基于提示调优的对齐方法，结合NLI分类器在对话任务的跨语言迁移中取得优于主流大模型的效果，尤其适配低资源少样本场景。
摘要：
针对高资源语言（如英语）训练的语言模型的跨语言迁移已在众多NLP任务中得到广泛研究，但针对对话任务的关注仍相当有限，部分原因在于获取非英语对话数据的成本高昂，导致数据覆盖范围有限。本研究提出用于跨语言对齐预训练的XSGD——这是一个并行的大规模多语言对话数据集，我们通过将仅含英语的Schema-Guided Dialogue (SGD) 数据集（Rastogi等人，2020）翻译成105种其他语言构建而成，XSGD每种语言包含约33万条话语。为生成对齐的跨语言表示，我们提出了一种基于提示调优的高效方法，用于学习对齐提示。我们还研究了两种不同的分类器：基于NLI的分类器和普通分类器，并测试了对齐提示所赋能的跨语言能力。我们在两个对话任务（槽填充和意图分类）上评估了模型的跨语言泛化能力。结果表明，基于NLI的分类器具备强大且高效的建模能力，我们的对齐提示实现了显著的跨语言迁移提升，尤其在少样本设置下。此外，我们强调，在零样本和少样本设置下，我们的方法相较于text-davinci-003、ChatGPT等大语言模型（LLM）表现更优。尽管LLM在英语中表现出色，但它们在其他语言（尤其是低资源语言）中的跨语言能力十分有限。
英文摘要：
Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks, but focus on conversational tasks has been rather limited. This is partly due to the high cost of obtaining non-English conversational data, which results in limited coverage. In this work, we introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset that we created by translating the English-only Schema-Guided Dialogue (SGD) dataset (Rastogi et al., 2020) into 105 other languages. XSGD contains approximately 330k utterances per language. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts. We also investigate two different classifiers: NLI-based and vanilla classifiers, and test cross-lingual capability enabled by the aligned prompts. We evaluate our model's cross-lingual generalization capabilities on two conversation tasks: slot-filling and intent classification. Our results demonstrate the strong and efficient modeling ability of NLI-based classifiers and the large cross-lingual transfer improvements achieved by our aligned prompts, particularly in few-shot settings. In addition, we highlight the nice results of our approach compared to LLMs such as text-davinci-003 and ChatGPT in both zero-shot and few-shot settings. While LLMs exhibit impressive performance in English, their cross-lingual capabilities in other languages, particularly low-resource languages, are limited.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
