arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.01388cs.CLcs.AI
收藏

如何建立用户模拟器来训练基于RL的对话系统

How to Build User Simulators to Train RL-based Dialog Systems

Carnegie Mellon University(卡内基梅隆大学);University of California, Davis(加州大学戴维斯分校)

Weiyan Shi, Kun Qian, Xuewei Wang, Zhou Yu

摘要:

用户模拟器对于训练基于强化学习(RL)的对话模型是必不可少的。仿真器的性能直接影响RL策略。然而,构建一个好的用户模拟器来模拟真实的用户行为是具有挑战性的。我们提出了一种标准化用户模拟器建设的方法,可用于社区使用相同的用户模拟器公平地比较对话系统的质量。我们提出了实现六个用户模拟器训练不同的对话规划和生成方法。然后,我们计算一组自动度量直接和间接地评估这些模拟器的质量。我们还要求人类用户通过对模拟对话进行评级并与经过训练的系统进行交互来直接或间接地评估模拟器。本文提出了一个全面的评估框架,用户模拟器的研究,并提供了一个更好的了解不同的用户模拟器的优点和缺点,以及它们对训练系统的影响。

英文摘要:

User simulators are essential for training reinforcement learning (RL) based dialog models. The performance of the simulator directly impacts the RL policy. However, building a good user simulator that models real user behaviors is challenging. We propose a method of standardizing user simulator building that can be used by the community to compare dialog system quality using the same set of user simulators fairly. We present implementations of six user simulators trained with different dialog planning and generation methods. We then calculate a set of automatic metrics to evaluate the quality of these simulators both directly and indirectly. We also ask human users to assess the simulators directly and indirectly by rating the simulated dialogs and interacting with the trained systems. This paper presents a comprehensive evaluation framework for user simulator study and provides a better understanding of the pros and cons of different user simulators, as well as their impacts on the trained systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑