奇妙的奖励及其驯服：任务导向对话系统中奖励学习的个案研究
Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systems
摘要：
当学习面向任务的对话（ToD）代理时，强化学习（RL）技术可以自然地用于训练对话策略以实现用户特定的目标。以往的研究主要集中在采用先进的强化学习技术来训练ToD智能体，而对奖励函数的设计研究较少。本文旨在回答如何有效地学习和利用奖励函数来训练端到端（E2E）ToD代理的问题。具体来说，我们引入两个广义目标奖励函数学习，经典的学习排名文献的启发。此外，我们利用学习的奖励函数来指导E2E ToD代理的训练。通过所提出的技术，我们在Multiwoz 2.0数据集上的E2E响应生成任务上取得了有竞争力的结果。源代码和检查点在https://github.com/Shentao-YANG/Fantastic_Reward_ICLR2023上公开发布。
英文摘要：
When learning task-oriented dialogue (ToD) agents, reinforcement learning (RL) techniques can naturally be utilized to train dialogue strategies to achieve user-specific goals. Prior works mainly focus on adopting advanced RL techniques to train the ToD agents, while the design of the reward function is not well studied. This paper aims at answering the question of how to efficiently learn and leverage a reward function for training end-to-end (E2E) ToD agents. Specifically, we introduce two generalized objectives for reward-function learning, inspired by the classical learning-to-rank literature. Further, we utilize the learned reward function to guide the training of the E2E ToD agent. With the proposed techniques, we achieve competitive results on the E2E response-generation task on the Multiwoz 2.0 dataset. Source code and checkpoints are publicly released at https://github.com/Shentao-YANG/Fantastic_Reward_ICLR2023.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
