通过带监督正则化的离线模仿学习复现人类复杂对话策略
Replicating Complex Dialogue Policy of Humans via Offline Imitation Learning with Supervised Regularization
AI总结：
针对任务型对话中模仿人类策略的难题，提出一种无需用户模拟器、利用状态转移信息并引入正则化技巧的离线模仿学习模型，在四个公开数据集的动作预测任务上取得更优性能。
摘要：
策略学习（PL）是任务型对话系统中的一个模块，用于训练智能体在每一轮对话中做出动作。模仿人类行为是策略学习的一个基本问题。然而，监督学习（SL）和强化学习（RL）框架都无法很好地模仿人类。训练RL模型需要与用户模拟器进行在线交互，而模拟复杂的人类策略是困难的。基于SL的模型的性能受到协变量偏移问题的限制。具体而言，对话是一个序贯决策过程，当前话语和动作的微小差异会导致后续话语的显著差异。因此，由于训练和测试对话数据的统计特征逐渐变得不同，SL模型的泛化能力受到限制。本研究提出了一种离线模仿学习模型，该模型从真实对话数据集中学习策略，并且不需要用户模拟器。它还利用状态转移信息，从而减轻了协变量偏移问题的影响。我们引入了一种正则化技巧，使我们的模型能够被有效优化。我们在四个独立的公开对话数据集上研究了我们模型的性能。实验结果表明，我们的模型在动作预测任务中表现更好。
英文摘要：
Policy learning (PL) is a module of a task-oriented dialogue system that trains an agent to make actions in each dialogue turn. Imitating human action is a fundamental problem of PL. However, both supervised learning (SL) and reinforcement learning (RL) frameworks cannot imitate humans well. Training RL models require online interactions with user simulators, while simulating complex human policy is hard. Performances of SL-based models are restricted because of the covariate shift problem. Specifically, a dialogue is a sequential decision-making process where slight differences in current utterances and actions will cause significant differences in subsequent utterances. Therefore, the generalize ability of SL models is restricted because statistical characteristics of training and testing dialogue data gradually become different. This study proposed an offline imitation learning model that learns policy from real dialogue datasets and does not require user simulators. It also utilizes state transition information, which alleviates the influence of the covariate shift problem. We introduced a regularization trick to make our model can be effectively optimized. We investigated the performance of our model on four independent public dialogue datasets. The experimental result showed that our model performed better in the action prediction task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
