arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2509.01167cs.CVcs.CLcs.LG

TempCore:视频问答基准测试是否具有时间感知性?一个帧选择敏感性分析与基准测试

TempCore: Are Video QA Benchmarks Temporally Grounded?

POSTECH(POSTECH大学)

Hyunjong Ok, Jaeho Lee

AI总结:

本文通过帧选择敏感性分析揭示视频问答基准测试中多数样本对帧选择不敏感,仅少数样本具有时间敏感性,提出TempCore用于评估时间敏感样本。

摘要:

视觉语言模型(VLMs)只能处理有限数量的视频帧,使得帧选择成为一种实际需求。但当前的视频问答基准测试是否真正需要时间敏感的帧选择,还是大多数问题无论展示哪些帧都能回答?我们引入了帧选择敏感性(FSS),这是一种针对每个样本的诊断方法,用于衡量当最相关帧被最不相关帧替代时,VLM准确性如何变化。在六个基准和八个VLM上,我们发现大多数样本对帧选择不敏感:只有少数样本真正对帧选择敏感。结合FSS与语言独立性分数(LIS)发现,仅8-33%的样本是时间敏感的。我们构建了TempCore,一种紧凑的评估子集,用于从现有基准中隔离这些时间敏感样本,并将在发表时发布代码和每样本注释。

英文摘要:

Vision-language models (VLMs) can ingest only a limited number of video frames, making frame selection a practical necessity. But do current Video QA benchmarks genuinely require temporal frame selection, or can most questions be answered regardless of which frames are shown? We introduce Frame Selection Sensitivity (FSS), a per-sample diagnostic that measures how much VLM accuracy changes when the most relevant frames are replaced with the least relevant ones. Across six benchmarks and eight VLMs, we find that a large majority of samples are frame-agnostic: only a minority are genuinely sensitive to frame choice. Combining FSS with a Language Independence Score (LIS) reveals that merely 5.5--31% of samples are Temporally Sensitive. We construct TempCore, compact evaluation subsets that isolate these temporal samples from existing benchmarks, and will release code and per-sample annotations upon publication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑