Hi Model，生成 'nice' 而不是 'good' 并不像生成 'rice' 那么糟糕！迈向上下文与语义融合的对话生成损失函数和评估指标
Hi Model, generating 'nice' instead of 'good' is not as bad as generating 'rice'! Towards Context and Semantic Infused Dialogue Generation Loss Function and Evaluation Metric
AI总结：
本文指出了传统基于词汇的对话生成指标缺乏语义和上下文考虑的局限，提出了融合上下文与语义的SemTextualLogue损失函数及Dialuation评估指标，实验证明该方法能提升模型性能且与人类评估相关性更强。
摘要：
在过去的二十年中，对话建模取得了显著进展，从简单的基于规则的响应转向了个性化和具有说服力的响应生成。然而，尽管取得了这些进步，对话生成的目标函数和评估指标却停滞不前。这些基于词汇的指标，例如交叉熵和 BLEU，存在两个关键限制：(a) 不考虑语义的逐词匹配：对于未能为 "good" 生成 "nice" 和 "rice" 的情况，它分配相同的失败惩罚；(b) 缺乏用于评估生成响应的上下文属性：即使生成的响应与当前的对话上下文相关，它仍可能因为不匹配语料库中提供的黄金话语而受到惩罚。在本文中，我们首先全面调查了这些限制，并提出了一种名为 Semantic Infused Contextualized diaLogue (SemTextualLogue) 的新损失函数。我们还制定了一个名为 Dialuation 的评估指标，结合了上下文和语义相关性。我们在两个对话语料库上对非预训练和预训练模型进行了实验，涵盖了面向任务和开放域场景。我们发现，使用 SemTextualLogue 损失训练的对话生成模型比传统的交叉熵损失函数获得了更好的性能。研究结果表明，对话生成模型的有效训练在很大程度上取决于语义和上下文的结合。这种模式也反映在引入的 Dialuation 指标中，与传统指标相比，同时考虑上下文和语义与人类评估的相关性更强。
英文摘要：
Over the past two decades, dialogue modeling has made significant strides, moving from simple rule-based responses to personalized and persuasive response generation. However, despite these advancements, the objective functions and evaluation metrics for dialogue generation have remained stagnant. These lexical-based metrics, e.g., cross-entropy and BLEU, have two key limitations: (a) word-to-word matching without semantic consideration: It assigns the same credit for failure to generate "nice" and "rice" for "good", (b) missing context attribute for evaluating the generated response: Even if a generated response is relevant to the ongoing dialogue context, it may still be penalized for not matching the gold utterance provided in the corpus. In this paper, we first investigate these limitations comprehensively and propose a new loss function called Semantic Infused Contextualized diaLogue (SemTextualLogue) loss function. We also formulate an evaluation metric called Dialuation, incorporating both context and semantic relevance. We experimented with both non-pretrained and pre-trained models on two dialogue corpora, encompassing task-oriented and open-domain scenarios. We found that the dialogue generation models trained with SemTextualLogueloss attained superior performance compared to the traditional cross-entropy loss function. The findings establish that the effective training of a dialogue generation model hinges significantly on incorporating semantics and context. This pattern is also mirrored in the introduced Dialuation metric, where the consideration of both context and semantics correlates more strongly with human evaluation compared to traditional metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
