VSTAR：面向场景与话题转换的情境语义理解的视频对话数据集
VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic Transitions
AI总结：
本文提出基于395部电视剧的大规模视频对话数据集VSTAR，并构建场景分割、话题分割与对话生成基准，验证多模态信息与片段对视频对话理解及生成的重要性。
摘要：
视频对话理解是一个具有挑战性的问题，它要求机器从弱对齐的视频和对话中提取的情境语义进行感知、解析和推理。现有的大多数基准将两种模态都视为帧独立的视觉理解任务，而忽略了多模态对话中的内在属性，例如场景和话题转换。在本文中，我们提出了视频情境场景与话题感知对话（VSTAR）数据集，这是一个基于395部电视剧的大规模视频对话理解数据集。基于VSTAR，我们提出了两个用于视频对话理解的基准：场景分割和话题分割，以及一个用于视频对话生成的基准。在这些基准上进行了全面的实验，以证明多模态信息和片段在视频对话理解与生成中的重要性。
英文摘要：
Video-grounded dialogue understanding is a challenging problem that requires machine to perceive, parse and reason over situated semantics extracted from weakly aligned video and dialogues. Most existing benchmarks treat both modalities the same as a frame-independent visual understanding task, while neglecting the intrinsic attributes in multimodal dialogues, such as scene and topic transitions. In this paper, we present Video-grounded Scene&Topic AwaRe dialogue (VSTAR) dataset, a large scale video-grounded dialogue understanding dataset based on 395 TV series. Based on VSTAR, we propose two benchmarks for video-grounded dialogue understanding: scene segmentation and topic segmentation, and one benchmark for video-grounded dialogue generation. Comprehensive experiments are performed on these benchmarks to demonstrate the importance of multimodal information and segments in video-grounded dialogue understanding and generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
