arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1709.04409cs.CL
收藏

社会对话系统评价技术综述

A Review of Evaluation Techniques for Social Dialogue Systems

Heriot-Watt University(赫瑞瓦特大学)

Amanda Cercas Curry, Helen Hastie, Verena Rieser

摘要:

与面向目标的对话相比,社会对话没有明确的衡量任务成功的标准。因此,对这些系统的评估是非常困难的。在本文中,我们回顾了当前的评估方法,重点是自动度量。我们的结论是,回合制的指标往往忽略了上下文,并没有考虑到几个答复是有效的,而对话结束的奖励主要是手工制作的。两者都缺乏人类感知的基础。

英文摘要:

In contrast with goal-oriented dialogue, social dialogue has no clear measure of task success. Consequently, evaluation of these systems is notoriously hard. In this paper, we review current evaluation methods, focusing on automatic metrics. We conclude that turn-based metrics often ignore the context and do not account for the fact that several replies are valid, while end-of-dialogue rewards are mainly hand-crafted. Both lack grounding in human perceptions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑