ChatZero:通过伪目标语言实现零样本跨语言对话生成
ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language
AI总结:
针对低资源语言零样本对话生成难题,提出基于跨语言语码转换的ChatZero模型,通过无监督对比学习缩小语义差距,在零样本下达到监督学习90%以上性能及最优基线水平。
摘要:
尽管大语言模型(LLMs)展现出惊人的能力,但在其众多令人瞩目的应用中,在其他低资源语言上的表现仍有不足。此外,现有大多数方法依赖大规模对话语料,因此在零样本场景下构建对话生成系统仍是一项相当大的挑战。为解决这一挑战,我们提出了一种基于跨语言语码转换方法的新型端到端零样本对话生成模型ChatZero。首先,我们构建带占位符的语码转换语言和伪目标语言。随后为实现跨语言语义迁移,我们采用无监督对比学习,在高维语义空间中缩小互为正样本的源语言、语码转换语言与伪目标语言之间的语义差距。在多语言DailyDialog和DSTC7-AVSD数据集上的实验表明,零样本设置下的ChatZero相比监督学习能达到其90%以上的性能,且与其他基线模型相比达到了当前最优性能。
英文摘要:
Although large language models(LLMs) show amazing capabilities, among various exciting applications discovered for LLMs fall short in other low-resource languages. Besides, most existing methods depend on large-scale dialogue corpora and thus building systems for dialogue generation in a zero-shot scenario remains a considerable challenge. To address this challenge, we propose a novel end-to-end zero-shot dialogue generation model ChatZero based on cross-lingual code-switching method. First, we construct code-switching language and pseudo-target language with placeholders. Then for cross-lingual semantic transfer, we employ unsupervised contrastive learning to minimize the semantics gap of the source language, code-switching language, and pseudo-target language that are mutually positive examples in the high dimensional semantic space. Experiments on the multilingual DailyDialog and DSTC7-AVSD datasets demonstrate that ChatZero can achieve more than 90\% of the original performance under the zero-shot case compared to supervised learning, and achieve state-of-the-art performance compared with other baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。