Cinéaste：一个细粒度的上下文电影问答基准
Cinéaste: A Fine-grained Contextual Movie Question Answering Benchmark
AI总结：
提出长篇电影理解基准Cinéaste，包含3119个问答对，通过两阶段过滤确保质量，实验揭示现有模型在长时程时间推理上存在瓶颈。
摘要：
尽管视觉语言模型的最新进展改善了视频理解，但诊断它们进行深度叙事理解的能力仍然是一项挑战。现有基准通常测试短片段识别或使用基于模板的问题，在评估对长篇叙事内容的细粒度推理方面留下了关键空白。为了解决这些空白，我们引入了 $\mathsf{Cin\acute{e}aste}$，一个用于长篇电影理解的全面基准。我们的数据集包含 3,119 个多选题问答对，源自 200 部不同电影的 1,805 个场景，涵盖五个新颖的细粒度上下文推理类别。我们使用 GPT-4o 通过整合视觉描述、字幕、场景标题和摘要来生成多样化、上下文丰富的问题，这些问题需要深度的叙事理解。为确保高质量评估，我们的流程包含两阶段过滤过程：上下文独立性过滤确保问题需要视频上下文，而上下文准确性过滤根据电影内容验证事实一致性，从而减轻幻觉。实验表明，现有的 MLLM 在 $\mathsf{Cin\acute{e}aste}$ 上表现挣扎；我们的分析揭示，长时程时间推理是主要瓶颈，表现最好的开源模型仅达到 63.15% 的准确率。这凸显了细粒度上下文理解方面的重大挑战，以及在长篇电影理解方面取得进展的必要性。
英文摘要：
While recent advancements in vision-language models have improved video understanding, diagnosing their capacity for deep, narrative comprehension remains a challenge. Existing benchmarks often test short-clip recognition or use template-based questions, leaving a critical gap in evaluating fine-grained reasoning over long-form narrative content. To address these gaps, we introduce $\mathsf{Cin\acute{e}aste}$, a comprehensive benchmark for long-form movie understanding. Our dataset comprises 3,119 multiple-choice question-answer pairs derived from 1,805 scenes across 200 diverse movies, spanning five novel fine-grained contextual reasoning categories. We use GPT-4o to generate diverse, context-rich questions by integrating visual descriptions, captions, scene titles, and summaries, which require deep narrative understanding. To ensure high-quality evaluation, our pipeline incorporates a two-stage filtering process: Context-Independence filtering ensures questions require video context, while Contextual Veracity filtering validates factual consistency against the movie content, mitigating hallucinations. Experiments show that existing MLLMs struggle on $\mathsf{Cin\acute{e}aste}$; our analysis reveals that long-range temporal reasoning is a primary bottleneck, with the top open-source model achieving only 63.15\% accuracy. This underscores significant challenges in fine-grained contextual understanding and the need for advancements in long-form movie comprehension.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
