arXiv 2603.299432026-07-13cs.CV

EC-Bench：超长视频的枚举与计数基准

Diagnosing Long-Video Quantitative Reasoning in Multimodal LLMs via Enumeration and Counting

The University of Tokyo, Japan（东京大学）；Institute of Science Tokyo, Japan（东京科学大学）；National Institute of Informatics, Japan（国立信息学研究所）

Fumihiko Tsuchiya, Taiki Miyanishi, Shunsuke Yasuki, Mahiro Ukai, Nakamasa Inoue, Shuhei Kurita, Yusuke Iwasawa, Yutaka Matsuo

备注：: The first two authors are equally contributed. The data and code are publicly available at: https://github.com/matsuolab/EC-Bench

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

EC-Bench针对超长视频的枚举与计数问题，通过152部超过30分钟的视频和1699个查询，评估多模态大语言模型的性能，揭示模型在时间推理和计数任务中的局限性。

摘要：

长视频中的计数仍是计算机视觉中的基础但研究不足的挑战。现实中的记录往往持续数十分钟甚至更长，包含稀疏且多样的事件，使长时间的时序推理特别困难。然而，大多数现有的视频计数基准仅关注短片段，并只评估最终的数值答案，提供很少关于应计什么或模型是否在时间上一致识别相关实例的见解。我们介绍了EC-Bench，一个联合评估长视频中枚举、计数和时间证据定位的基准。EC-Bench包含152部超过30分钟的视频和1,699个与显式证据跨度配对的查询。在22个多模态大语言模型（MLLMs）上，最佳模型在枚举任务上的准确率为29.98%，在计数任务上的准确率为23.74%，而人类表现分别达到78.57%和82.97%。我们的分析揭示了枚举准确性、时间定位和计数性能之间的强相关性。这些结果突显了当前MLLMs的根本局限性，并确立了EC-Bench作为长形式定量视频推理的具有挑战性的基准。

英文摘要：

Final-answer video QA can show whether a model predicts the right number, but not which instances it counted, when the supporting evidence occurs, or why it failed. We diagnose long-video quantitative reasoning in multimodal large language models (MLLMs) through three coupled abilities: enumerating query-relevant instances, temporally grounding supporting evidence, and aggregating the evidence into counts. To support this analysis, we build EC-Bench, an evidence-annotated evaluation suite with 152 untrimmed videos longer than 30 minutes, 1,699 open-ended queries across six reasoning categories, and human-verified evidence spans. We evaluate 22 open-source and proprietary MLLMs using timestamped visual frames and transcripts. The best average scores reach only 29.98% Enumeration F1 and 23.74% Counting accuracy, compared with human performance of 78.57% and 82.97%, respectively. Our analyses show that counting errors are rarely isolated arithmetic mistakes: Enumeration F1 is strongly associated with Counting accuracy, temporal grounding quality is associated with lower counting error, and Counting accuracy drops as supporting evidence becomes more distributed. These findings recast long-video counting as evidence retrieval, temporal grounding, deduplication, and aggregation across the video, rather than simple numerical prediction.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。