NarrativeTrack: 评估实体中心推理在叙事理解中的表现
NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding
AI总结：
NarrativeTrack通过细粒度实体中心推理评估多模态大语言模型的叙事理解能力，揭示了感知接地与时间推理之间的根本权衡。
摘要：
多模态大语言模型（MLLMs）在视觉-语言推理方面取得了显著进展，但其在视频中时间展开叙事的理解能力仍待探索。真正的叙事理解需要明确谁在做什么、何时、何地，并在动态视觉和时间背景下保持一致的实体表示。我们引入NarrativeTrack，首个评估MLLMs叙事理解的基准，通过细粒度实体中心推理进行评估。与现有仅限于短片段或粗粒度场景级语义的基准不同，我们分解视频为构成实体，并通过结构化评估框架Compositional Reasoning Progression（CRP）检查其连续性，该框架在三个维度上逐步增加叙事复杂性：实体存在、实体变化和实体歧义。CRP挑战模型从时间持续性发展到情境演变和细粒度感知推理。全自动的实体中心流水线使可扩展地提取时间接地的实体表示，为CRP提供基础。对最新MLLMs的评估显示，模型在视觉转换和时间动态中难以稳健跟踪实体，常在情境变化下产生身份幻觉。开源通用MLLMs表现出强感知接地但弱时间一致性，而视频专用MLLMs捕捉时间上下文但产生实体上下文幻觉。这些发现揭示了感知接地与时间推理之间的根本权衡，表明叙事理解仅从它们的整合中产生。NarrativeTrack提供了首个系统框架，以诊断和推进MLLMs中时间接地的叙事理解。
英文摘要：
Multimodal large language models (MLLMs) have achieved impressive progress in vision-language reasoning, yet their ability to understand temporally unfolding narratives in videos remains underexplored. True narrative understanding requires grounding who is doing what, when, and where, maintaining coherent entity representations across dynamic visual and temporal contexts. We introduce NarrativeTrack, the first benchmark to evaluate narrative understanding in MLLMs through fine-grained entity-centric reasoning. Unlike existing benchmarks limited to short clips or coarse scene-level semantics, we decompose videos into constituent entities and examine their continuity via a Compositional Reasoning Progression (CRP), a structured evaluation framework that progressively increases narrative complexity across three dimensions: entity existence, entity changes, and entity ambiguity. CRP challenges models to advance from temporal persistence to contextual evolution and fine-grained perceptual reasoning. A fully automated entity-centric pipeline enables scalable extraction of temporally grounded entity representations, providing the foundation for CRP. Evaluations of state-of-the-art MLLMs reveal that models fail to robustly track entities across visual transitions and temporal dynamics, often hallucinating identity under context shifts. Open-source general-purpose MLLMs exhibit strong perceptual grounding but weak temporal coherence, while video-specific MLLMs capture temporal context yet hallucinate entities' contexts. These findings uncover a fundamental trade-off between perceptual grounding and temporal reasoning, indicating that narrative understanding emerges only from their integration. NarrativeTrack provides the first systematic framework to diagnose and advance temporally grounded narrative comprehension in MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
