arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.12637cs.CL
收藏

哦,我的错误!:包括转回话语的朝向真实对话状态跟踪

Oh My Mistake!: Toward Realistic Dialogue State Tracking including Turnback Utterances

Korea University(高丽大学);Electronics and Telecommunications Research Institute(电子通信研究院);Sejong University(世宗大学)

Takyoung Kim, Yukyung Lee, Hoonsang Yoon, Pilsung Kang, Junseong Bang, Misuk Kim

摘要:

对话状态跟踪(DST)是端到端会话系统的关键组件,其主要目的是构建一个能够很好地响应真实世界情况的模型。虽然我们经常在普通对话中不时改变主意,但当前的基准数据集并不能充分反映这种情况,而是由过度简化的对话组成,其中没有人在对话中改变主意。作为激发本研究的主要问题,“当前的基准数据集是否足够多样化,以处理在某个主题结束后改变主意的随意对话?我们发现答案是“否”,因为当基于模板的turnback话语被注入数据集时,DST模型不能参考以前的用户偏好。即使在最简单的思维改变(折返)场景中,DST模型的性能也会显著退化。然而,我们发现,当在训练集中明确设计折返场景时,这种性能退化可以恢复,这意味着问题不在于DST模型,而在于基准数据集的构建。

英文摘要:

The primary purpose of dialogue state tracking (DST), a critical component of an end-to-end conversational system, is to build a model that responds well to real-world situations. Although we often change our minds from time to time during ordinary conversations, current benchmark datasets do not adequately reflect such occurrences and instead consist of over-simplified conversations, in which no one changes their mind during a conversation. As the main question inspiring the present study, "Are current benchmark datasets sufficiently diverse to handle casual conversations in which one changes their mind after a certain topic is over?" We found that the answer is "No" because DST models cannot refer to previous user preferences when template-based turnback utterances are injected into the dataset. Even in the the simplest mind-changing (turnback) scenario, the performance of DST models significantly degenerated. However, we found that this performance degeneration can be recovered when the turnback scenarios are explicitly designed in the training set, implying that the problem is not with the DST models but rather with the construction of the benchmark dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑