MORE:一个多语言文档解析基准和评估
MORE: A Multilingual Document Parsing Benchmark and Evaluation
AI总结:
研究针对多语言文档解析,现有基准侧重高资源语言。为此引入MORE基准,其具规模大、结构复杂、数据真实的特点,用它评估模型,为长尾语言建性能基线并验证基准有效性。
摘要:
多语言文档包含丰富文化、科学发现和历史记录。将其解析为结构化、机器可读格式对获取全球知识至关重要。现有基准主要关注英语和中文等高资源语言,对其他语言模型性能评估存在盲点。我们引入MORE,一个用于多语言文档解析评估的大规模基准,具有前所未有的规模、结构复杂性和数据真实性,并评估了模型。
英文摘要:
Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。