视觉与语言中缺失的一块：漫画理解综述
One missing piece in Vision and Language: A Survey on Comics Understanding
AI总结：
本文综述漫画理解的数据、任务与方法，提出 LoCU 任务分类框架，并指出视觉-语言模型在漫画领域的挑战与未来方向。
摘要：
视觉-语言模型近期已发展为多功能系统，能够在文档理解、视觉问答和定位等一系列任务中取得高性能，且通常处于零样本设置。漫画理解是一个复杂且多面的领域，有望从这些进展中大幅受益。漫画作为一种媒介，结合了丰富的视觉与文本叙事，通过连续画格向AI模型提出跨越图像分类、目标检测、实例分割以及更深层叙事理解的任务挑战。然而，漫画的独特结构——以风格、阅读顺序和非线性叙事的创造性变化为特征——带来了一组不同于其他视觉-语言领域的挑战。在本综述中，我们从数据集和任务两个角度对漫画理解进行全面回顾。我们的贡献有五方面：（1）分析漫画媒介的结构，详细说明其独特的构成元素；（2）调研漫画研究中广泛使用的数据集和任务，强调它们在推动该领域发展中的作用；（3）提出 Layer of Comics Understanding（LoCU）框架，这是一种重新定义漫画内视觉-语言任务并为未来工作奠定基础的新型分类法；（4）按照 LoCU 框架对现有方法进行详细回顾和分类；（5）最后，突出当前研究挑战并提出未来探索方向，尤其是在视觉-语言模型应用于漫画的背景下。本综述首次提出面向任务的漫画智能框架，旨在通过解决数据可用性和任务定义方面的关键空白来指导未来研究。与本综述相关的项目可在 https://github.com/emanuelevivoli/awesome-comics-understanding 获取。
英文摘要：
Vision-language models have recently evolved into versatile systems capable of high performance across a range of tasks, such as document understanding, visual question answering, and grounding, often in zero-shot settings. Comics Understanding, a complex and multifaceted field, stands to greatly benefit from these advances. Comics, as a medium, combine rich visual and textual narratives, challenging AI models with tasks that span image classification, object detection, instance segmentation, and deeper narrative comprehension through sequential panels. However, the unique structure of comics -- characterized by creative variations in style, reading order, and non-linear storytelling -- presents a set of challenges distinct from those in other visual-language domains. In this survey, we present a comprehensive review of Comics Understanding from both dataset and task perspectives. Our contributions are fivefold: (1) We analyze the structure of the comics medium, detailing its distinctive compositional elements; (2) We survey the widely used datasets and tasks in comics research, emphasizing their role in advancing the field; (3) We introduce the Layer of Comics Understanding (LoCU) framework, a novel taxonomy that redefines vision-language tasks within comics and lays the foundation for future work; (4) We provide a detailed review and categorization of existing methods following the LoCU framework; (5) Finally, we highlight current research challenges and propose directions for future exploration, particularly in the context of vision-language models applied to comics. This survey is the first to propose a task-oriented framework for comics intelligence and aims to guide future research by addressing critical gaps in data availability and task definition. A project associated with this survey is available at https://github.com/emanuelevivoli/awesome-comics-understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
