DSTC 11 Track 4 开放域对话系统鲁棒且多语言的自动评估指标概述
Overview of Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems at DSTC 11 Track 4
AI总结：
本文概述了DSTC11 Track 4，旨在推动开放域对话系统在多领域、多维度和多语言下可靠的自动评估指标，介绍了提供的数据集、基线及两个子任务的提交与结果。
摘要：
神经网络的出现与快速发展彻底改变了对话系统的研究，并随之引发了关于其自动评估的各种挑战。开放域对话系统的自动评估作为一个开放性挑战，一直是众多研究人员关注的中心。尽管为提升自动指标与人工评估的相关性做出了持续努力，但对其在多个领域和维度上的鲁棒性进行评估的尝试却寥寥无几。此外，它们的焦点主要集中在英语语言上。所有这些挑战促使了在各个领域、维度和语言中均可靠的自动评估指标的开发。第11届对话系统技术挑战赛（DSTC11）中的此赛道是推动鲁棒且多语言自动评估指标的持续努力的一部分。本文描述了提供给参与者的数据集与基线，并讨论了两个提出子任务的提交与结果细节。
英文摘要：
The advent and fast development of neural networks have revolutionized the research on dialogue systems and subsequently have triggered various challenges regarding their automatic evaluation. Automatic evaluation of open-domain dialogue systems as an open challenge has been the center of the attention of many researchers. Despite the consistent efforts to improve automatic metrics' correlations with human evaluation, there have been very few attempts to assess their robustness over multiple domains and dimensions. Also, their focus is mainly on the English language. All of these challenges prompt the development of automatic evaluation metrics that are reliable in various domains, dimensions, and languages. This track in the 11th Dialogue System Technology Challenge (DSTC11) is part of the ongoing effort to promote robust and multilingual automatic evaluation metrics. This article describes the datasets and baselines provided to participants and discusses the submission and result details of the two proposed subtasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
