MangaVQA和MangaLMM：多模态漫画理解的基准和专用模型
MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal Manga Understanding
AI总结：
本文提出MangaVQA和MangaLMM，用于多模态漫画理解的基准和专用模型，通过视觉问答和文本识别任务提升漫画叙事理解能力。
摘要：
漫画，或日本漫画，是一种复杂的多模态叙事形式，将图像和文本以复杂的方式结合在一起。教大型多模态模型（LMMs）以人类水平理解此类叙事，可以帮助漫画创作者反思和改进他们的故事。为此，我们介绍了两个多模态漫画理解的基准：MangaOCR，旨在针对页面内的文本识别，以及MangaVQA，一个新的基准，旨在通过视觉问答评估上下文理解。MangaVQA包含526个高质量的手工构建的问题-答案对，能够在多样化的叙事和视觉场景中实现可靠的评估。基于这些基准，我们开发了MangaLMM，一个专门针对漫画的模型，从开源LMM Qwen2.5-VL微调而来，以同时处理两项任务。通过广泛的实验，包括与专有模型如GPT-4o和Gemini 2.5的比较，我们评估了LMMs理解漫画的能力。我们的基准和模型为在丰富叙事领域中评估和推进LMMs提供了全面的基础。
英文摘要：
Manga, or Japanese comics, is a richly multimodal narrative form that blends images and text in complex ways. Teaching large multimodal models (LMMs) to understand such narratives at a human-like level could help manga creators reflect on and refine their stories. To this end, we introduce two benchmarks for multimodal manga understanding: MangaOCR, which targets in-page text recognition, and MangaVQA, a novel benchmark designed to evaluate contextual understanding through visual question answering. MangaVQA consists of 526 high-quality, manually constructed question-answer pairs, enabling reliable evaluation across diverse narrative and visual scenarios. Building on these benchmarks, we develop MangaLMM, a manga-specialized model finetuned from the open-source LMM Qwen2.5-VL to jointly handle both tasks. Through extensive experiments, including comparisons with proprietary models such as GPT-4o and Gemini 2.5, we assess how well LMMs understand manga. Our benchmark and model provide a comprehensive foundation for evaluating and advancing LMMs in the richly narrative domain of manga.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
