文档解析距离解决还有多远？PureDocBench：一个跨干净、退化和现实场景的可追溯基准测试
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings
AI总结：
本文提出PureDocBench，一个可追溯的基准测试，覆盖10个领域、66个子类和1475页文档，包含清洁、数字退化和现实退化三种版本。评估40种模型发现，文档解析尚未解决，最佳模型得分仅74分，公式识别仍是瓶颈，通用VLM在退化下表现不如专用模型。
摘要：
过去一年间，已有超过20种开源文档解析模型，但该领域仍几乎只在OmniDocBench上进行基准测试，这是一个1355页手动标注的数据集，其最高分数已超过90%。我们运行的三阶段审核流程对OmniDocBench的21353个评估块进行了审查，确认了2580个错误（12.08%）；结合一年以上的公开可用性，标注质量和污染风险都对其排名提出了质疑。为了解决这些问题，我们提出了PureDocBench，一个程序生成、可追溯的基准测试，从HTML/CSS渲染文档图像，并从同一来源生成可验证的标注，涵盖10个领域、66个子类和1475页，每页在三种版本中：清洁、数字退化和现实退化（总计4425张图像）。评估40种模型，涵盖管道专家、端到端专家和通用VLM，我们发现：（i）文档解析远未解决：最佳模型得分仅约74分，最强和最弱模型之间有44.6分的差距；（ii）参数不超过4B的专用解析器与通用VLM（大5-100倍）相当或更优，但公式识别仍是共同瓶颈，没有任何模型在三个赛道平均公式指标上超过67%；（iii）通用VLM在数字/现实退化下仅损失0.99/8.52总分，而管道专家损失4.90/14.21，产生排名反转，使仅清洁评估对部署来说误导。所有数据、代码和制品均已公开发布。
英文摘要：
Document parsing models are improving rapidly, but existing benchmarks are still insufficient for a comprehensive evaluation of their capabilities. We introduce PureDocBench, a benchmark with 10 domains, 66 subcategories, and 1,475 pages. Each page has matched Clean, Digital, and Real versions, covering clean renders, simulated degradation, and real capture or sharing conditions. The resulting 4,425 images have source-derived annotations for text, formulas, tables, and reading order, checked through automated validation and human review. Keeping page content and annotations fixed allows direct comparisons across image conditions. We evaluate 58 models, including pipeline-based parsers, end-to-end parsers, and general vision-language models. The best three-track average score is 79.69 out of 100, and the mean across models is 63.84. Scores drop by 3.34 points on Digital and 11.61 points on Real on average, relative to Clean. TeleOCR leads on Clean, GLM-5.3-Flash on Digital, and Gemini 3.6 Flash on Real. The benchmark dataset, source documents, generation tools, and evaluation code are publicly available at https://github.com/zhihengli-casia/PureDocBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
