AvgOut：一种简单的消除迟钝响应的输出概率测度
AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses
摘要：
许多序列对序列的对话模型往往会产生安全的、没有信息的响应。已经作出各种有益的努力，试图消除这些障碍。然而，这些方法要么在推理过程中改进解码算法，要么依赖于手工制作的功能，要么采用复杂的模型。在我们的工作中，我们构建了对话模型，这些模型动态地知道哪些话语或令牌是枯燥的，而无需任何功能工程。具体来说，我们从一个简单而有效的自动度量AvgOut开始，它计算训练期间解码器端所有时间步长的平均输出概率分布。该度量直接估计哪些令牌更有可能被生成，从而使其成为模型多样性的忠实评估（即，对于不同的模型，令牌概率应该更均匀地分布，而不是在几个沉闷的令牌处达到峰值）。然后，我们利用这个新的指标，提出了三个模型，促进多样性，而不会失去相关性。第一个模型MinAvgOut通过每个批次的输出分布直接最大化多样性得分;第二个模型LFT（Label Fine-Tuning）在源序列中预先添加一个由多样性得分连续缩放的标签以控制多样性水平;第三个模型RL采用强化学习并将多样性得分视为奖励信号。此外，我们通过结合MinAvgOut和RL的损失项来实验混合模型。所有四个模型在多样性和相关性方面都大大优于其基础LSTM-RNN模型，并且与竞争基线相当或更好（也通过人工评估进行了验证）。此外，我们的方法是正交的基础模型，使它们适用于作为一个附加到其他新兴的更好的对话模型在未来。
英文摘要：
Many sequence-to-sequence dialogue models tend to generate safe, uninformative responses. There have been various useful efforts on trying to eliminate them. However, these approaches either improve decoding algorithms during inference, rely on hand-crafted features, or employ complex models. In our work, we build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. Specifically, we start with a simple yet effective automatic metric, AvgOut, which calculates the average output probability distribution of all time steps on the decoder side during training. This metric directly estimates which tokens are more likely to be generated, thus making it a faithful evaluation of the model diversity (i.e., for diverse models, the token probabilities should be more evenly distributed rather than peaked at a few dull tokens). We then leverage this novel metric to propose three models that promote diversity without losing relevance. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch; the second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level; the third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal. Moreover, we experiment with a hybrid model by combining the loss terms of MinAvgOut and RL. All four models outperform their base LSTM-RNN model on both diversity and relevance by a large margin, and are comparable to or better than competitive baselines (also verified via human evaluation). Moreover, our approaches are orthogonal to the base model, making them applicable as an add-on to other emerging better dialogue models in the future.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
