Re:Verse -- 能读懂漫画吗？
Re:Verse -- Can Your VLM Read a Manga?
AI总结：
本文通过分析漫画叙事理解，揭示现有VLM在时间因果和跨面板连贯性上的不足，提出新的评估框架，系统研究长篇叙事理解能力。
摘要：
当前视觉语言模型（VLMs）在处理连续视觉叙事时，存在表层识别与深层叙事推理之间的关键差距。通过全面研究漫画叙事理解，我们发现尽管近期大型多模态模型在单格解读上表现优异，但系统性地在时间因果和跨格连贯性上失败，这是连贯故事理解的核心要求。我们引入了一个新的评估框架，结合细粒度多模态标注、跨模态嵌入分析和检索增强评估，系统地表征这些限制。我们的方法包括（i）严格的标注协议，通过对齐的轻小说文本将视觉元素与叙事结构联系起来；（ii）在多个推理范式下进行综合评估，包括直接推理和检索增强生成；（iii）跨模态相似性分析，揭示当前VLM联合表示中的根本性不一致。将此框架应用于Re:Zero漫画11章中的308个标注面板，我们通过生成叙事、上下文对话基础和时间推理三个核心评估轴，进行了首次系统研究长篇叙事理解。我们的发现表明，当前模型缺乏真正的故事级智能，尤其在非线性叙事、角色一致性及长序列因果推理上表现不佳。本文为评估叙事智能奠定了基础，并提供了深度序列理解离散视觉叙事能力的可行见解。
英文摘要：
Current Vision Language Models (VLMs) demonstrate a critical gap between surface-level recognition and deep narrative reasoning when processing sequential visual storytelling. Through a comprehensive investigation of manga narrative understanding, we reveal that while recent large multimodal models excel at individual panel interpretation, they systematically fail at temporal causality and cross-panel cohesion, core requirements for coherent story comprehension. We introduce a novel evaluation framework that combines fine-grained multimodal annotation, cross-modal embedding analysis, and retrieval-augmented assessment to systematically characterize these limitations. Our methodology includes (i) a rigorous annotation protocol linking visual elements to narrative structure through aligned light novel text, (ii) comprehensive evaluation across multiple reasoning paradigms, including direct inference and retrieval-augmented generation, and (iii) cross-modal similarity analysis revealing fundamental misalignments in current VLMs' joint representations. Applying this framework to Re:Zero manga across 11 chapters with 308 annotated panels, we conduct the first systematic study of long-form narrative understanding in VLMs through three core evaluation axes: generative storytelling, contextual dialogue grounding, and temporal reasoning. Our findings demonstrate that current models lack genuine story-level intelligence, struggling particularly with non-linear narratives, character consistency, and causal inference across extended sequences. This work establishes both the foundation and practical methodology for evaluating narrative intelligence, while providing actionable insights into the capability of deep sequential understanding of Discrete Visual Narratives beyond basic recognition in Multimodal Models. Project Page: https://re-verse.vercel.app
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
