VDialogUE：一个面向视觉基础对话的统一评估基准
VDialogUE: A Unified Evaluation Benchmark for Visually-grounded Dialogue
AI总结：
针对视觉基础对话系统缺乏标准化评估框架的问题，提出统一评估基准VDialogUE，定义五大任务覆盖六个数据集，基于层次分析法设计VDscore指标，并构建两阶段预训练基线模型VISIT以推动该领域发展。
摘要：
视觉基础对话系统融合了文本和视觉输入等多种通信模式，已成为日益热门的研究领域。然而，缺乏标准化的评估框架对该领域的发展评估构成了挑战。为此，我们提出了VDialogUE，一个用于统一评估的视觉基础对话基准。它定义了五个核心多模态对话任务，并涵盖六个数据集。此外，为了全面评估模型在所有任务上的性能，我们开发了一种名为VDscore的新型评估指标，该指标基于层次分析法（AHP）方法。另外，我们提出了一个简单但高效的基线模型，名为VISIT（视觉基础对话Transformer），以推动通用多模态对话系统的发展。它通过两阶段预训练策略逐步构建其多模态基础和对话能力。我们相信，VDialogUE基准以及评估脚本和我们的基线模型将加速视觉基础对话系统的发展，并促成更复杂、更有效的预训练模型的开发。
英文摘要：
Visually-grounded dialog systems, which integrate multiple modes of communication such as text and visual inputs, have become an increasingly popular area of investigation. However, the absence of a standardized evaluation framework poses a challenge in assessing the development of this field. To this end, we propose \textbf{VDialogUE}, a \textbf{V}isually-grounded \textbf{Dialog}ue benchmark for \textbf{U}nified \textbf{E}valuation. It defines five core multi-modal dialogue tasks and covers six datasets. Furthermore, in order to provide a comprehensive assessment of the model's performance across all tasks, we developed a novel evaluation metric called VDscore, which is based on the Analytic Hierarchy Process~(AHP) method. Additionally, we present a straightforward yet efficient baseline model, named \textbf{VISIT}~(\textbf{VIS}ually-grounded d\textbf{I}alog \textbf{T}ransformer), to promote the advancement of general multi-modal dialogue systems. It progressively builds its multi-modal foundation and dialogue capability via a two-stage pre-training strategy. We believe that the VDialogUE benchmark, along with the evaluation scripts and our baseline models, will accelerate the development of visually-grounded dialog systems and lead to the development of more sophisticated and effective pre-trained models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
