MPDocBench-Parse:面向实际的多页文档解析基准测试
MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing
AI总结:
针对现有基准测试在真实场景中评估不足的问题,提出MPDocBench-Parse基准,包含433份多页文档(3246页),覆盖15种文档类型,设计全面的内容保真度和逻辑结构评估协议,实验表明现有模型在语义连续性、视觉内容解析和层次结构恢复方面存在明显局限。
摘要:
文档解析将视觉丰富的文档转换为机器可读的结构化表示,为信息系统提供了关键基础。尽管已有许多文档解析基准测试,但它们仍不足以应对真实场景。现有基准测试要么专注于特定任务,要么仅评估单页、以文本为中心的设置,因此不足以处理实际的多页解析。此外,它们缺乏对语义连续性、层次结构恢复和视觉内容保留的细粒度评估。为解决这些不足,我们提出了MPDocBench-Parse,一个面向实际应用的多页文档解析基准测试。它包含433份人工标注的文档,共3246页,覆盖中英文15种文档类型,具有多样化的布局风格,并支持文档级端到端评估。我们进一步设计了一套全面的内容保真度和逻辑结构评估协议,涵盖文本、表格和公式识别,截断文本和表格合并,图形提取,阅读顺序以及标题层次恢复。实验表明,尽管现有模型在基本文本提取方面表现良好,但在语义连续性整合、视觉内容解析和层次结构恢复方面仍存在明显局限。MPDocBench-Parse为将文档解析推进到更真实的场景提供了统一基础。
英文摘要:
Document parsing converts visually rich documents into machine-readable structured representations, forming a crucial foundation for information systems. Although many benchmarks have been proposed for document parsing, they remain inadequate for realistic scenarios. Existing benchmarks either focus on specific tasks or assess only single-page, text-centric settings, making them insufficient for practical multi-page parsing. Moreover, they lack fine-grained evaluation of semantic continuity, hierarchical structure recovery, and visual content preservation. To address these gaps, we propose MPDocBench-Parse, a benchmark for multi-page document parsing in real-world applications. It contains 433 manually annotated documents with 3,246 pages, covering 15 document types in English and Chinese, with diverse layout styles, and supports document-level end-to-end evaluation. We further design a comprehensive protocol for content fidelity and logical structure, covering text, table, and formula recognition, truncated text and table merging, figure extraction, reading order, and heading hierarchy recovery. Experiments show that, while existing models perform well on basic text extraction, they still suffer clear limitations in semantic continuity integration, visual content parsing, and hierarchical structure recovery. MPDocBench-Parse provides a unified foundation for advancing document parsing toward more realistic scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。