MDPBench：多语言文档解析的实际场景基准
MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios
AI总结：
本文提出MDPBench，首个多语言数字和照片文档解析基准，包含17种语言的3400张文档图像，揭示闭源模型在非拉丁文和真实场景下的性能优势，开放源代码以促进更包容的解析系统发展。
摘要：
我们介绍了多语言文档解析基准（MDPBench），这是首个针对多语言数字和拍摄文档解析的基准。文档解析在清洁、数字化且格式良好的页面上取得了显著进展，但几乎只在少数主导语言上进行。目前没有系统性的基准来评估模型在不同脚本和低资源语言上的数字和拍摄文档表现。MDPBench包含3,400张文档图像，涵盖17种语言、多样化的脚本和多样的拍摄条件，高质量注释通过严格的人工标注、手动校正和人工验证流程产生。为确保公平比较和防止数据泄漏，我们维护了公开和私有评估分割。对开源和闭源模型的全面评估揭示了一个显著发现：虽然闭源模型（特别是Gemini3-Pro）相对稳健，但开源替代品在非拉丁文和真实场景下的性能急剧下降，拍摄文档平均下降17.8%，非拉丁文脚本下降14.0%。这些结果揭示了语言和条件间的显著性能不平衡，并指出了构建更包容、可部署的解析系统的方向。源代码可在https://github.com/Yuliang-Liu/MultimodalOCR上获取。
英文摘要：
We introduce Multilingual Document Parsing Benchmark, the first benchmark for multilingual digital and photographed document parsing. Document parsing has made remarkable strides, yet almost exclusively on clean, digital, well-formatted pages in a handful of dominant languages. No systematic benchmark exists to evaluate how models perform on digital and photographed documents across diverse scripts and low-resource languages. MDPBench comprises 3,400 document images spanning 17 languages, diverse scripts, and varied photographic conditions, with high-quality annotations produced through a rigorous pipeline of expert model labeling, manual correction, and human verification. To ensure fair comparison and prevent data leakage, we maintain separate public and private evaluation splits. Our comprehensive evaluation of both open-source and closed-source models uncovers a striking finding: while closed-source models (notably Gemini3-Pro) prove relatively robust, open-source alternatives suffer dramatic performance collapse, particularly on non-Latin scripts and real-world photographed documents, with an average drop of 17.8% on photographed documents and 14.0% on non-Latin scripts. These results reveal significant performance imbalances across languages and conditions, and point to concrete directions for building more inclusive, deployment-ready parsing systems. Source available at https://github.com/Yuliang-Liu/MultimodalOCR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
