arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.11357cs.CL
收藏

用于提高基于检索的开放域对话系统性能的自注意比较模块

Self-attention Comparison Module for Boosting Performance on Retrieval-based Open-Domain Dialog Systems

Beijing Institute of Technology(北京理工大学);Huazhong University of Science and Technology(华中科技大学)

Tian Lan, Xian-Ling Mao, Zhipeng Zhao, Wei Wei, Heyan Huang

摘要:

由于预先训练好的语言模型的广泛应用,基于检索的开放领域对话系统近年来受到了研究者的广泛关注。以往的工作大多只根据查询与每个候选响应之间的匹配程度来选择合适的响应。虽然已经取得了良好的性能,这些最近的工作忽略了候选响应之间的比较,这可以提供丰富的信息,以选择最合适的反应。直觉上,当模型可以访问所有候选响应之间的比较信息时,可以做出更好的决策。为了利用候选响应之间的比较信息,在本文中,我们提出了一种新的和插件的自注意力比较模块检索为基础的开放域对话系统,称为SCM。大量的实验结果表明,我们提出的自我注意力比较模块有效地提高了现有的基于检索的开放域对话系统的性能。此外,我们已经公开了我们的源代码,以供未来研究。

英文摘要:

Since the pre-trained language models are widely used, retrieval-based open-domain dialog systems, have attracted considerable attention from researchers recently. Most of the previous works select a suitable response only according to the matching degree between the query and each individual candidate response. Although good performance has been achieved, these recent works ignore the comparison among the candidate responses, which could provide rich information for selecting the most appropriate response. Intuitively, better decisions could be made when the models can get access to the comparison information among all the candidate responses. In order to leverage the comparison information among the candidate responses, in this paper, we propose a novel and plug-in Self-attention Comparison Module for retrieval-based open-domain dialog systems, called SCM. Extensive experiment results demonstrate that our proposed self-attention comparison module effectively boosts the performance of the existing retrieval-based open-domain dialog systems. Besides, we have publicly released our source codes for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑