arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.04250cs.CL
收藏

非自回归神经对话生成

Non-Autoregressive Neural Dialogue Generation

Department of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);ShannonAI(ShannonAI(香侬人工智能))

Qinghong Han, Yuxian Meng, Fei Wu, Jiwei Li

摘要:

最大互信息(MMI),其对响应($y$)和上下文($x$)之间的双向依赖性进行建模,即,前向概率$\log p(y|x)$和后向概率$\log p(x|y)$已被广泛用作\sts模型中的目标,以解决开放域对话生成中的迟钝响应问题。不幸的是,在\sts模型的框架下,从$\log p(y|x) + \log p(x|y)$直接解码是不可行的,因为第二部分(即,$p(x|y)$)需要在计算目标之前完成目标生成,并且用于$y$的搜索空间是巨大的。根据经验,首先在给定$p(y|x)$的情况下生成N最佳列表,然后使用$p(x|y)$对N最佳列表进行重新排序,这不可避免地导致非全局最优解。在本文中,我们建议使用非自回归(非AR)生成模型来解决这个非全局最优性问题。由于在非AR生成中独立地生成目标令牌,因此每个目标词的$p(x|y)$可以在其生成时立即计算,并且不必等待整个序列的完成。这自然解决了解码中的非全局最优问题。实验结果表明,所提出的非AR策略产生更多样化,连贯性和适当的反应,产生实质性的收益,在BLEU分数和人类的评价。

英文摘要:

Maximum Mutual information (MMI), which models the bidirectional dependency between responses ($y$) and contexts ($x$), i.e., the forward probability $\log p(y|x)$ and the backward probability $\log p(x|y)$, has been widely used as the objective in the \sts model to address the dull-response issue in open-domain dialog generation. Unfortunately, under the framework of the \sts model, direct decoding from $\log p(y|x) + \log p(x|y)$ is infeasible since the second part (i.e., $p(x|y)$) requires the completion of target generation before it can be computed, and the search space for $y$ is enormous. Empirically, an N-best list is first generated given $p(y|x)$, and $p(x|y)$ is then used to rerank the N-best list, which inevitably results in non-globally-optimal solutions. In this paper, we propose to use non-autoregressive (non-AR) generation model to address this non-global optimality issue. Since target tokens are generated independently in non-AR generation, $p(x|y)$ for each target word can be computed as soon as it's generated, and does not have to wait for the completion of the whole sequence. This naturally resolves the non-global optimal issue in decoding. Experimental results demonstrate that the proposed non-AR strategy produces more diverse, coherent, and appropriate responses, yielding substantive gains in BLEU scores and in human evaluations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑