CC-OCR：用于评估大型多模态模型读写能力的全面且具挑战性的 OCR 基准
CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy
AI总结：
本文提出包含四大赛道、39 个子集和 7,058 张标注图像的 CC-OCR 基准，用于系统评估九个主流 LMMs 的 OCR 读写能力并揭示其关键短板。
摘要：
大型多模态模型（LMMs）在依据自然语言指令识别文档图像方面已展现出令人瞩目的性能。然而，在面对丰富结构与细粒度视觉挑战时，其读写能力究竟达到何种程度仍不清楚。当前领域缺乏一个能够有效衡量 LMMs 读写能力的全面基准；现有基准往往受限于狭窄场景和特定任务。为此，我们提出 CC-OCR，这是一个涵盖多样场景、任务和挑战的全面基准。CC-OCR 包含四个以 OCR 为核心的赛道：多场景文本读取、多语言文本读取、文档解析和关键信息抽取。它包括 39 个子集，共 7,058 张完整标注图像，其中 41% 来源于真实应用，并且是首次公开发布。我们评估了九个主流 LMMs，揭示了这些模型的优势与不足，尤其是在文本定位、多方向文本以及重复幻觉方面。CC-OCR 旨在全面评估 LMMs 在以 OCR 为中心的任务上的能力，推动这一关键领域的持续进步。
英文摘要：
Large Multimodal Models (LMMs) have demonstrated impressive performance in recognizing document images with natural language instructions. However, it remains unclear to what extent capabilities in literacy with rich structure and fine-grained visual challenges. The current landscape lacks a comprehensive benchmark to effectively measure the literate capabilities of LMMs. Existing benchmarks are often limited by narrow scenarios and specified tasks. To this end, we introduce CC-OCR, a comprehensive benchmark that possesses a diverse range of scenarios, tasks, and challenges. CC-OCR comprises four OCR-centric tracks: multi-scene text reading, multilingual text reading, document parsing, and key information extraction. It includes 39 subsets with 7,058 full annotated images, of which 41% are sourced from real applications, and released for the first time. We evaluate nine prominent LMMs and reveal both the strengths and weaknesses of these models, particularly in text grounding, multi-orientation, and hallucination of repetition. CC-OCR aims to comprehensively evaluate the capabilities of LMMs on OCR-centered tasks, facilitating continued progress in this crucial area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
