arXiv 2606.013932026-05-31cs.CLcs.AIcs.CV

Dr. DocBench：专家级与困难文档解析的综合基准

Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing

Stanford University（斯坦福大学）；MIT（麻省理工学院）；Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）；Harvard University（哈佛大学）；IBM Research（IBM研究院）；University of Arizona（亚利桑那大学）；Duke University（杜克大学）；UC Berkeley（加州大学伯克利分校）；LMU Munich（慕尼黑路德维希-马克西米利安大学）

Minglai Yang, Xinyan Velocity Yu, Pengyuan Li, Xinyu Guo, Zhenting Qi, Konwoo Kim, Longtian Ye, Xiaolong Luo, Jinhe Bi, Henry Zhang, Haris Riaz, Xuan Zhang, Yun… 展开作者

Minglai Yang, Xinyan Velocity Yu, Pengyuan Li, Xinyu Guo, Zhenting Qi, Konwoo Kim, Longtian Ye, Xiaolong Luo, Jinhe Bi, Henry Zhang, Haris Riaz, Xuan Zhang, Yunze Xiao, Bangya Liu, Tom Tang, Yunfei Zhao, Qunshu Lin, Zihan Wang, Minghao Liu, Michael Lingzhi Li, Yilun Du, Jesse Thomason, Rogerio Feris, Alex Pentland, Zexue He

备注：: 27 pages, 13 figures, 14 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出Dr. DocBench基准，通过基于解析器失败的采样从多语言书籍语料库中选取挑战性文档，包含52个BISAC主题领域和65k高质量标注，用于评估专家级文档解析能力。

摘要：

文档解析和识别是视觉语言模型（VLM）和文档处理系统的基本能力。然而，现有的光学字符识别（OCR）和文档解析基准在覆盖范围和难度上日益受限：许多基准专注于常见文档类型或均匀采样的页面，现代解析器在这些页面上已表现良好，而对专家领域结构（如化学公式、乐谱、复杂表格和跨页布局）的标注有限。我们引入了Dr. DocBench，一个面向专家级文档解析的难度感知基准。Dr. DocBench基于大规模多语言书籍语料库构建，涵盖52个BISAC主题领域，并通过基于解析器失败的采样选择挑战性文档，针对多个最先进系统难以处理的案例。它包含来自平均约100页的长文档的4,514个标注页面，具有65k高质量的页面级和块级标注，涵盖布局、阅读顺序、层次关系和特定领域的视觉内容。对基于流水线的解析器和通用VLM的评估表明，在现有基准上的强性能并不能迁移到我们的专家级文档解析中。我们的分析揭示了跨主题、内容类型和结构属性的重大失败，突显了Dr. DocBench作为诊断和推进文档智能的综合测试平台的作用。

英文摘要：

Document parsing and recognition are fundamental capabilities for vision-language models (VLMs) and document processing systems. However, existing Optical Character Recognition (OCR) and document parsing benchmarks are increasingly limited in coverage and difficulty: many focus on common document genres or uniformly sampled pages where modern parsers already perform strongly, while offering limited annotation for expert-domain structures such as chemical formula, music notation, complex tables, and cross-page layouts. We introduce Dr. DocBench, a difficulty-aware benchmark for expert-level document parsing. Built from a large-scale multilingual book corpus, Dr. DocBench spans 52 BISAC subject domains and selects challenging documents through parser-failure-based sampling, targeting cases where multiple state-of-the-art systems struggle. It contains 4,514 annotated pages from long documents averaging around 100 pages, with 65k high-quality page- and block-level annotations for layout, reading order, hierarchical relations, and domain-specific visual contents. Evaluations of pipeline-based parsers and general-purpose VLMs show that strong performance on existing benchmarks does not transfer to our expert-level document parsing. Our analysis reveals substantial failures across subjects, content types, and structural attributes, highlighting Dr. DocBench as a comprehensive testbed for diagnosing and advancing document intelligence.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。