Real5-OmniDocBench:面向野外鲁棒文档解析的全尺度物理重建基准
Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild
AI总结:
提出首个对OmniDocBench v1.5进行全尺度一对一物理重建的基准Real5-OmniDocBench,覆盖扫描、弯曲、屏幕摄影、光照和倾斜五种真实场景,通过完整真值映射实现性能退化的因子级归因,揭示文档解析中的“现实鸿沟”。
摘要:
尽管视觉语言模型(VLM)在OmniDocBench等数字文档基准上取得了近乎完美的分数,但由于缺乏受控且真实的评估,它们在不可预测的物理世界中的性能仍然很大程度上未知。我们引入了Real5-OmniDocBench,这是第一个对整个OmniDocBench v1.5(1355张图像)在五个关键真实场景(扫描、弯曲、屏幕摄影、光照和倾斜)中进行全尺度、一对一物理重建的基准。与之前要么缺乏数字对应关系要么采用部分采样的基准不同,我们完整的真值映射首次实现了严格的因子级性能退化归因——使我们能够精确定位失败是源于几何畸变、光学伪影还是模型限制。我们的基准为社区设定了一个具有挑战性的新标准,表明文档解析中的“现实鸿沟”远未消除,并提供了一个诊断工具来指导真正鲁棒的文档智能的发展。
英文摘要:
While Vision-Language Models (VLMs) achieve near-perfect scores on digital document benchmarks like OmniDocBench, their performance in the unpredictable physical world remains largely unknown due to the lack of controlled yet realistic evaluations. We introduce Real5-OmniDocBench, the first benchmark that performs a full-scale, one-to-one physical reconstruction of the entire OmniDocBench v1.5 (1,355 images) across five critical real-world scenarios: Scanning, Warping, Screen-Photography, Illumination, and Skew. Unlike prior benchmark that either lack digital correspondence or employ partial sampling, our complete ground-truth mapping enables, for the first time, rigorous factor-wise attribution of performance degradation-allowing us to pinpoint whether failures stem from geometric distortions, optical artifacts, or model limitations. Our benchmark establishes a challenging new standard for the community, demonstrating that the 'reality gap' in document parsing is far from closed, and provides a diagnostic tool to guide the development of truly resilient document intelligence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。