arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1606.08340cs.CL
收藏

主题感知神经响应生成

Topic Aware Neural Response Generation

Nankai University(南开大学);Microsoft Research(微软研究院);Beihang University(北京航空航天大学)

Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma

摘要:

我们考虑将主题信息纳入序列到序列框架中,为聊天机器人生成信息丰富且有趣的响应。为此,我们提出了一个主题感知的序列到序列(TA-Seq 2Seq)模型。该模型利用话题模拟人的先验知识,引导人在会话中形成信息丰富、有趣的回答,并通过联合注意力机制和有偏生成概率来利用话题信息生成会话。联合注意力机制通过消息注意力将输入消息的隐藏向量概括为上下文向量,从预先训练好的LDA模型获得的消息的主题词中,通过主题注意力合成主题向量,并让这些向量在解码时共同影响词的生成。为了增加主题词出现在响应中的可能性,该模型通过添加额外的概率项来改变主题词的生成概率,以使总体分布偏向。对自动评价指标和人工注释的实证研究表明,TA-Seq 2Seq可以生成更多信息和有趣的响应,并显着优于最先进的响应生成模型。

英文摘要:

We consider incorporating topic information into the sequence-to-sequence framework to generate informative and interesting responses for chatbots. To this end, we propose a topic aware sequence-to-sequence (TA-Seq2Seq) model. The model utilizes topics to simulate prior knowledge of human that guides them to form informative and interesting responses in conversation, and leverages the topic information in generation by a joint attention mechanism and a biased generation probability. The joint attention mechanism summarizes the hidden vectors of an input message as context vectors by message attention, synthesizes topic vectors by topic attention from the topic words of the message obtained from a pre-trained LDA model, and let these vectors jointly affect the generation of words in decoding. To increase the possibility of topic words appearing in responses, the model modifies the generation probability of topic words by adding an extra probability item to bias the overall distribution. Empirical study on both automatic evaluation metrics and human annotations shows that TA-Seq2Seq can generate more informative and interesting responses, and significantly outperform the-state-of-the-art response generation models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑