arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2304.12982cs.CL

DSTC 11 任务型对话赛道:从对话中归纳意图

Intent Induction from Conversations for Task-Oriented Dialogue Track at DSTC 11

AWS AI Labs(AWS人工智能实验室)

James Gung, Raphael Shu, Emily Moeng, Wesley Rose, Salvatore Romeo, Yassine Benajiba, Arshit Gupta, Saab Mansour, Yi Zhang

AI总结:

本文介绍 DSTC 11 的对话意图归纳赛道,构建真实客服场景基准、两项子任务、三个数据集与基线,并汇总34个团队的提交结果。

摘要:

随着虚拟助理的需求和采用不断增长,近期研究探索了通过自动归纳意图,或归纳槽位与对话状态来加速机器人模式设计的方法。然而,由于缺乏专门的基准和标准化评估,研究进展难以追踪,系统之间也难以比较。该挑战赛道作为第十一届对话系统技术挑战(Eleventh Dialog Systems Technology Challenge)的一部分,提出了一个基准,旨在在人工客服与客户之间客户服务交互的真实场景中,评估自动归纳客户意图的方法。我们提出了两个子任务,用于逐步解决意图的自动归纳问题,并给出相应的评估方法。随后,我们介绍了三个适合评估这些任务的数据集,并提出了简单基线。最后,我们总结了该挑战赛道的提交情况与结果,共收到来自34个团队的提交。

英文摘要:

With increasing demand for and adoption of virtual assistants, recent work has investigated ways to accelerate bot schema design through the automatic induction of intents or the induction of slots and dialogue states. However, a lack of dedicated benchmarks and standardized evaluation has made progress difficult to track and comparisons between systems difficult to make. This challenge track, held as part of the Eleventh Dialog Systems Technology Challenge, introduces a benchmark that aims to evaluate methods for the automatic induction of customer intents in a realistic setting of customer service interactions between human agents and customers. We propose two subtasks for progressively tackling the automatic induction of intents and corresponding evaluation methodologies. We then present three datasets suitable for evaluating the tasks and propose simple baselines. Finally, we summarize the submissions and results of the challenge track, for which we received submissions from 34 teams.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑