ViCaS：一个结合整体与像素级视频理解的数据集：基于字幕与定位分割
ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentation
AI总结：
本文提出ViCaS数据集及基准，通过提供带有详细字幕和像素级定位分割掩码的视频，统一了视频整体理解与像素级分割任务，并提出能有效同时处理这两项任务的模型架构。
摘要：
多模态大语言模型（MLLMs）的最新进展扩展了视频理解的研究，主要集中在视频字幕生成和问答等高级任务上。同时，少量工作致力于密集、像素精确的分割任务，通常涉及类别引导或基于指称的对象分割。尽管这两个方向对于开发具备人类级视频理解能力的模型都至关重要，但它们在很大程度上独立发展，拥有不同的基准和架构。本文旨在通过引入ViCaS来统一这些工作，这是一个新数据集，包含数千个具有挑战性的视频，每个视频都标注了详细的手写字幕，以及具有短语定位的多个对象的时序一致且像素精确的掩码。我们的基准在整体/高级理解和像素精确分割上评估模型。我们还提出了一个有效的模型架构，能够同时处理这两项任务。实验表明，我们的模型在ViCaS基准上显著优于现有方法。
英文摘要：
Recent advances in multimodal large language models (MLLMs) have expanded research in video understanding, primarily focusing on high-level tasks such as video captioning and question-answering. Meanwhile, a smaller body of work addresses dense, pixel-precise segmentation tasks, which typically involve category-guided or referral-based object segmentation. Although both directions are essential for developing models with human-level video comprehension, they have largely evolved separately, with distinct benchmarks and architectures. This paper aims to unify these efforts by introducing ViCaS, a new dataset containing thousands of challenging videos, each annotated with detailed, human-written captions and temporally consistent, pixel-accurate masks for multiple objects with phrase grounding. Our benchmark evaluates models on both holistic/high-level understanding and language-guided, pixel-precise segmentation. We also present carefully validated evaluation measures and propose an effective model architecture that can tackle our benchmark. Project page: https://ali2500.github.io/vicas-project/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
