arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.06849cs.CL
收藏

面向自主学习的端到端任务型对话系统

Toward Self-learning End-to-End Task-Oriented Dialog Systems

The Chinese University of Hong Kong(香港中文大学);Microsoft Research(微软研究院)

Xiaoying Zhang, Baolin Peng, Jianfeng Gao, Helen Meng

摘要:

端到端的任务机器人通常是通过静态且通常大小有限的语料库学习的。然而,当部署在动态、变化和开放的环境中与用户交互时,任务机器人在遇到偏离训练语料库的数据时往往会失败,即,非分销样本。在本文中,我们研究的问题,自动适应任务机器人不断变化的环境中学习,从人机交互与最小或零人类注释。我们提出了SL-AGENT,一种新的自学习框架,用于构建端到端的任务机器人。SL-AGENT由一个对话模型和一个预先训练好的奖励模型组成,用来预测Agent响应的质量。它使任务机器人能够通过使用集成的奖励模型通过强化学习从部署后积累的未标记的人类机器人对话日志中学习来自动适应不断变化的环境。四个良好的研究对话任务的实验结果表明,SL-代理自动适应不断变化的环境,使用自动和人工评估的有效性。我们将发布代码和数据以供进一步研究。

英文摘要:

End-to-end task bots are typically learned over a static and usually limited-size corpus. However, when deployed in dynamic, changing, and open environments to interact with users, task bots tend to fail when confronted with data that deviate from the training corpus, i.e., out-of-distribution samples. In this paper, we study the problem of automatically adapting task bots to changing environments by learning from human-bot interactions with minimum or zero human annotations. We propose SL-AGENT, a novel self-learning framework for building end-to-end task bots. SL-AGENT consists of a dialog model and a pre-trained reward model to predict the quality of an agent response. It enables task bots to automatically adapt to changing environments by learning from the unlabeled human-bot dialog logs accumulated after deployment via reinforcement learning with the incorporated reward model. Experimental results on four well-studied dialog tasks show the effectiveness of SL-AGENT to automatically adapt to changing environments, using both automatic and human evaluations. We will release code and data for further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑