SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series
IRIT, University of Toulouse, France（法国图卢兹大学IRIT中心）；Agency for Science, Technology and Research (A*STAR), Singapore（新加坡科技研究局）；CNRS, IRIT, France（法国CNRS与IRIT）
Galann Pennec, Zhengyuan Liu, Nicholas Asher, Philippe Muller, Nancy F. Chen
We introduce SagaQA, a long-form video benchmark for multi-hop reasoning over full-length TV series. Existing video reasoning benchmarks often emphasize local understanding of adjacent frames or clips. SagaQA addresses this gap by requiring high-level comprehension of extended multimodal narratives in entire TV shows. A distinguishing feature of SagaQA is the granularity of its reasoning steps. Our dataset necessitates long-range reasoning hops to connect information across completely different episodes. This requires models to reason over entire events and actions, demanding a deep understanding of the show's narration and progression at a multimodal level. Motivated by recent progress in agentic methods, we further study how different planning strategies handle such complex reasoning. We categorize these approaches into three classes-Parallel, Sequential, and Hybrid planners-and evaluate their ability to generate coherent and complete reasoning plans. Our results on SagaQA suggest that hybrid planners consistently produce higher-quality plans and exhibit stronger capabilities for complex, high-level narrative understanding in TV shows.
From Variance to Invariance: Qualitative Content Analysis for Narrative Graph Annotation
Department of Computer Science, University of Hamburg（汉堡大学计算机科学系）；Department of Socioeconomics, University of Hamburg（汉堡大学社会经济系）；AI and Explanability Group, Leuphana University of Lüneburg（吕内堡大学人工智能与可解释性小组）
NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens
Westlake University（西湖大学）；University of Waterloo（多伦多大学）；Hangzhou Normal University（杭州师范大学）；Shanghai AI Lab（上海人工智能实验室）；SJTU（上海交通大学）；NYU Shanghai（纽约大学上海分校）
