MMDocBench：面向细粒度视觉文档理解的大视觉语言模型基准测试
MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding
AI总结：
针对现有基准对大视觉语言模型细粒度视觉理解评估不足的问题，研究构建了含15类任务、4338个问答对的MMDocBench文档理解基准，测试了16款先进LVLM的相关能力并将公开资源。
摘要：
大视觉语言模型（LVLMs）已在诸多视觉-语言任务中取得出色表现，但其在细粒度视觉理解方面的能力仍未得到充分评估。现有基准要么包含的细粒度评估样本有限且与其他数据混合，要么仅局限于自然图像中的物体级评估。为全面评估LVLMs的细粒度视觉理解能力，我们提出利用具备多粒度、多模态信息的文档图像来补充自然图像评估。基于此，我们构建了MMDocBench——一个包含多种无需OCR（光学字符识别）的文档理解任务的基准，用于评估细粒度视觉感知与推理能力。MMDocBench定义了15项主要任务，包含4338个问答对和11353个支撑区域，覆盖研究论文、收据、财务报告、维基百科表格、图表、信息图等各类文档图像。基于MMDocBench，我们使用13个开源及3个专有先进LVLM开展了大量实验，评估了它们在不同任务和文档图像类型上的优缺点。该基准、任务说明及评估代码将公开发布。
英文摘要：
Large Vision-Language Models (LVLMs) have achieved remarkable performance in many vision-language tasks, yet their capabilities in fine-grained visual understanding remain insufficiently evaluated. Existing benchmarks either contain limited fine-grained evaluation samples that are mixed with other data, or are confined to object-level assessments in natural images. To holistically assess LVLMs' fine-grained visual understanding capabilities, we propose using document images with multi-granularity and multi-modal information to supplement natural images. In this light, we construct MMDocBench, a benchmark with various OCR-free document understanding tasks for the evaluation of fine-grained visual perception and reasoning abilities. MMDocBench defines 15 main tasks with 4,338 QA pairs and 11,353 supporting regions, covering various document images such as research papers, receipts, financial reports, Wikipedia tables, charts, and infographics. Based on MMDocBench, we conduct extensive experiments using 13 open-source and 3 proprietary advanced LVLMs, assessing their strengths and weaknesses across different tasks and document image types. The benchmark, task instructions, and evaluation code will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
