arXiv 2304.078492023-05-15cs.CL

ChatPLUG：面向数字人的互联网增强指令微调开放域生成式对话系统

ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human

DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）

Junfeng Tian, Hehong Chen, Guohai Xu, Ming Yan, Xing Gao, Jianhai Zhang, Chenliang Li, Jiayi Liu, Wenshen Xu, Haiyang Xu, Qi Qian, Wei Wang, Qinghao Ye, Jiejing… 展开作者

Junfeng Tian, Hehong Chen, Guohai Xu, Ming Yan, Xing Gao, Jianhai Zhang, Chenliang Li, Jiayi Liu, Wenshen Xu, Haiyang Xu, Qi Qian, Wei Wang, Qinghao Ye, Jiejing Zhang, Ji Zhang, Fei Huang, Jingren Zhou

备注：: 36 pages

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

ChatPLUG通过互联网增强指令微调构建中文开放域对话系统，融合知识、个性、记忆与共情能力，在自动和人工评估中优于现有系统，并已部署至实际应用。

摘要：

在本文中，我们提出了ChatPLUG，一个面向数字人应用的中文开放域对话系统，该系统以统一的互联网增强格式对广泛的对话任务进行指令微调。与其他专注于大规模预训练和扩大模型规模或对话语料的开放域对话模型不同，我们旨在通过互联网增强的指令微调，构建一个功能强大且实用的数字人对话系统，使其具备多样化的技能和良好的多任务泛化能力。为此，我们首先在通用文档语料和对话数据上进行大规模预训练，并采用课程学习策略，从而为ChatPLUG注入丰富的世界知识和对话能力。然后，我们收集了涵盖知识、个性、多轮记忆和共情等多样化特征的广泛对话任务，并通过统一的自然语言指令模板对这些任务进行指令微调。在指令微调过程中，我们还使用来自互联网搜索的外部知识，以缓解知识幻觉问题。我们表明，ChatPLUG在自动评估和人工评估中均优于最先进的中文对话系统，并在各种文本理解和生成任务上展现出强大的多任务泛化能力。此外，我们将ChatPLUG部署到实际应用中，如智能音箱和即时通讯应用，并实现快速推理。我们的模型和代码将在ModelScope（https://modelscope.cn/models/damo/ChatPLUG-3.7B）和Github（https://github.com/X-PLUG/ChatPLUG）上公开发布。

英文摘要：

In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format. Different from other open-domain dialogue models that focus on large-scale pre-training and scaling up model size or dialogue corpus, we aim to build a powerful and practical dialogue system for digital human with diverse skills and good multi-task generalization by internet-augmented instruction tuning. To this end, we first conduct large-scale pre-training on both common document corpus and dialogue data with curriculum learning, so as to inject various world knowledge and dialogue abilities into ChatPLUG. Then, we collect a wide range of dialogue tasks spanning diverse features of knowledge, personality, multi-turn memory, and empathy, on which we further instruction tune \modelname via unified natural language instruction templates. External knowledge from an internet search is also used during instruction finetuning for alleviating the problem of knowledge hallucinations. We show that \modelname outperforms state-of-the-art Chinese dialogue systems on both automatic and human evaluation, and demonstrates strong multi-task generalization on a variety of text understanding and generation tasks. In addition, we deploy \modelname to real-world applications such as Smart Speaker and Instant Message applications with fast inference. Our models and code will be made publicly available on ModelScope: https://modelscope.cn/models/damo/ChatPLUG-3.7B and Github: https://github.com/X-PLUG/ChatPLUG .

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。