arXiv 2605.013912026-06-12cs.CV

VISTA：视频交互时空分析基准

VISTA: Video Interaction Spatio-Temporal Analysis Benchmark

University of Central Florida（中央佛罗里达大学）；BITS Pilani（比特斯理工学院）；Ho Chi Minh City University of Science（胡志明市科学大学）；Amazon GenAI Project（亚马逊生成人工智能项目）

Alejandro Aparcedo, Akash Kumar, Aaryan Garg, Dalton Pham, Wen-Kai Chen, Anirudh Bharadwaj, Aman Chadha, Yogesh Rawat

备注：: Accepted to CVPR 2026 Workshop on Pixel-level Video Understanding in the Wild (PVUW)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出VISTA基准，通过分解视频为实体、动作和关系，实现开放集多实体多动作的时空理解评估，揭示传统指标掩盖的偏差。

摘要：

现有的视觉-语言模型（VLM）基准主要评估简单单动作视频、封闭属性集和受限实体类型的时空理解，未能捕捉真实世界视频理解中多样实体之间的自由形式多动作交互。此外，缺乏一个系统性的框架来分析模型在互补时空轴上的失败，阻碍了全面评估。为解决这些问题，我们引入了VISTA，一个视频交互时空分析基准，专为VLM中的开放集、多实体和多动作时空理解设计。VISTA将视频分解为可解释的实体、其关联动作和关系动态，实现多轴诊断以及关系、空间和时间理解的统一评估。我们的基准将多个数据集整合到一个单一的交互感知分类法中，包含约12K个精心策划的视频-查询对，涵盖多样场景和复杂性。我们在VISTA上系统评估了11个最先进的VLM，并分解了跨分类法的聚合性能，揭示了传统指标掩盖的缺陷和显著的时空偏差。通过在具有挑战性的数据集上提供详细的、分类法驱动的诊断，VISTA提供了一个精细的框架来指导模型设计、预训练策略和评估协议的进步。总体而言，VISTA是第一个大规模、交互感知的VLM时空理解诊断基准。

英文摘要：

Existing benchmarks for Vision-Language Models (VLMs) primarily evaluate spatio-temporal understanding on simple single-action videos, closed attribute sets and restricted entity types, failing to capture the freeform, multi-action interactions between diverse entities which characterize real-world video understanding. Furthermore, the lack of a systematic framework for analyzing model failures across complementary spatio-temporal axes hinders comprehensive evaluation. To address these gaps, we introduce VISTA, a Video Interaction Spatio-Temporal Analysis benchmark designed for open-set, multi-entity and multi-action spatio-temporal understanding in VLMs. VISTA decomposes videos into interpretable entities, their associated actions, and relational dynamics, enabling multi-axis diagnostics and unified assessment of relational, spatial, and temporal understanding. Our benchmark integrates multiple datasets into a single interaction-aware taxonomy and comprises ~12K curated video-query pairs spanning diverse scenes and complexities. We systematically evaluate 11 state-of-the-art VLMs on VISTA, and break down aggregate performance across our taxonomy to reveal shortcomings and pronounced spatio-temporal biases obscured by traditional metrics. By providing detailed, taxonomy-driven diagnostics on a challenging dataset, VISTA offers a nuanced framework to guide advances in model design, pretraining strategies, and evaluation protocols. Overall, VISTA is the first, large-scale, interaction-aware diagnostic benchmark for spatio-temporal understanding in VLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。