arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.08149cs.CL
收藏

TransferTransfo:一种基于神经网络的会话Agent的迁移学习方法

TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents

HuggingFace Inc.(HuggingFace公司);Prospect St.(81 Prospect St.(81 Prospect街));Brooklyn, New York 11201, USA(纽约布鲁克林区(美国)11201)

Thomas Wolf, Victor Sanh, Julien Chaumond, Clement Delangue

摘要:

我们介绍了一种新的生成数据驱动对话系统(例如聊天机器人)的方法,称为TransferTransfo,它是基于迁移学习的训练方案和高容量Transformer模型的组合。微调是通过使用一个多任务的目标,它结合了几个无监督的预测任务。由此产生的微调模型显示出对当前最先进的端到端会话模型(如内存增强seq 2seq和信息检索模型)的强大改进。在会话智能挑战2的私人持有的PERSONA-CHAT数据集上,该方法获得了新的最先进的技术,其困惑度、Hits@1和F1指标分别为16.28(45%绝对改善)、80.7(46%绝对改善)和19.5(20%绝对改善)。

英文摘要:

We introduce a new approach to generative data-driven dialogue systems (e.g. chatbots) called TransferTransfo which is a combination of a Transfer learning based training scheme and a high-capacity Transformer model. Fine-tuning is performed by using a multi-task objective which combines several unsupervised prediction tasks. The resulting fine-tuned model shows strong improvements over the current state-of-the-art end-to-end conversational models like memory augmented seq2seq and information-retrieval models. On the privately held PERSONA-CHAT dataset of the Conversational Intelligence Challenge 2, this approach obtains a new state-of-the-art, with respective perplexity, Hits@1 and F1 metrics of 16.28 (45 % absolute improvement), 80.7 (46 % absolute improvement) and 19.5 (20 % absolute improvement).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑