TOC-Bench：一个用于视频大语言模型的时序物体一致性基准
TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
AI总结：
TOC-Bench旨在评估视频大语言模型的时序物体一致性，通过设计三层时序必要性过滤协议，筛选出17900个时间依赖项，构建了包含2323个高质量问答对的基准，揭示了模型在事件计数、事件排序等任务上的不足。
摘要：
视频大语言模型（Video-LLMs）在通用视频理解方面取得了显著进展，但其保持时间物体一致性的能力仍待探索。现有基准测试往往侧重于事件识别、动作理解或粗粒度时间推理，而很少测试模型是否能保持同一物体在遮挡、消失、重现、状态转换和跨物体交互中的身份、状态和连续性。我们引入了TOC-Bench，一个用于评估视频大语言模型时间物体一致性的诊断基准。TOC-Bench是基于物体跟踪的：每个查询的主题都链接到每帧轨迹和结构化的时间事件时间线。为确保问题需要时间有序的视觉证据而不是语言先验、单帧捷径或无序帧提示，我们设计了三层时间必要性过滤协议，该协议删除了60.7%的候选QA对，保留了17,900个跨10个诊断维度的时间依赖项。从这个池子中，我们构建了一个经过人类验证的基准，包含2,323个高质量QA对，覆盖1,951个视频。在代表性Video-LLMs上的实验表明，时间物体一致性仍然是一个主要未解决的挑战，即使模型在通用视频理解基准上表现良好，也存在明显的事件计数、事件排序、身份敏感推理和hallucination-aware验证方面的弱点。这些结果表明，以物体为中心的时间一致性是当前Video-LLMs的关键瓶颈，而TOC-Bench提供了一个专注于诊断和改进物体感知时间推理的平台。资源可在https://github.com/cjzcjz666/toc_bench.git获取。
英文摘要：
Video large language models (Video-LLMs) have made strong progress in general video understanding, but their ability to maintain temporal object consistency remains underexplored. Existing benchmarks often emphasize event recognition, action understanding, or coarse temporal reasoning, while rarely testing whether models can preserve the identity, state, and continuity of the same object across occlusion, disappearance, reappearance, state transitions, and cross-object interactions. We introduce TOC-Bench, a diagnostic benchmark for evaluating temporal object consistency in Video-LLMs. TOC-Bench is object-track grounded: each queried subject is linked to a per-frame trajectory and a structured temporal event timeline. To ensure that questions require temporally ordered visual evidence rather than language priors, single-frame shortcuts, or unordered frame cues, we design a three-layer temporal-necessity filtering protocol, which removes 60.7% of candidate QA pairs and retains 17,900 temporally dependent items across 10 diagnostic dimensions. From this pool, we construct a human-verified benchmark with 2,323 high-quality QA pairs over 1,951 videos. Experiments on representative Video-LLMs show that temporal object consistency remains a major unsolved challenge, with notable weaknesses in event counting, event ordering, identity-sensitive reasoning, and hallucination-aware verification, even when models perform well on general video understanding benchmarks. These results suggest that object-centric temporal coherence is a key bottleneck for current Video-LLMs, and that TOC-Bench provides a focused platform for diagnosing and improving object-aware temporal reasoning. The resource is available at https://github.com/cjzcjz666/toc_bench.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
