arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.02240cs.CL
收藏

对话动作感知Transformer用于对话策略学习

Dialog Action-Aware Transformer for Dialog Policy Learning

The Chinese University of Hong Kong(香港中文大学);Tencent(腾讯)

Huimin Wang, Wai-Chung Kwan, Kam-Fai Wong

AI总结:

本文提出对话动作感知Transformer(DaTrans),利用预训练语言模型知识并设计掩码最后动作任务,加速强化学习对话策略学习,经模拟器与人工评估验证有效。

摘要:

近期工作通常通过训练强化学习(RL)智能体来确定最佳对话动作,以解决对话策略学习(DPL)问题。然而,现有的深度强化学习方法需要大量的智能体与用户交互才能达到可接受的性能。在本文中,我们提出充分利用预训练语言模型中的纯文本知识来加速RL智能体的学习速度。具体而言,我们设计了一个对话动作感知Transformer编码器(DaTrans),它集成了一种名为“掩码最后动作任务”的新微调过程,以促使DaTrans具备对话感知能力并提取动作特定特征。随后,DaTrans在强化学习设置中通过持续交互进一步优化,并在对话动作空间中进行探索,以最大化长期累积奖励。通过模拟器评估和人工评估,我们验证了所提模型的有效性和效率。

英文摘要:

Recent works usually address Dialog policy learning DPL by training a reinforcement learning (RL) agent to determine the best dialog action. However, existing works on deep RL require a large volume of agent-user interactions to achieve acceptable performance. In this paper, we propose to make full use of the plain text knowledge from the pre-trained language model to accelerate the RL agent's learning speed. Specifically, we design a dialog action-aware transformer encoder (DaTrans), which integrates a new fine-tuning procedure named masked last action task to encourage DaTrans to be dialog-aware and distils action-specific features. Then, DaTrans is further optimized in an RL setting with ongoing interactions and evolves through exploration in the dialog action space toward maximizing long-term accumulated rewards. The effectiveness and efficiency of the proposed model are demonstrated with both simulator evaluation and human evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑