ViDoRe V3：对复杂现实场景中检索增强生成的综合评估
ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarios
AI总结：
ViDoRe V3是一个多模态检索增强生成基准，涵盖10个专业领域数据集，通过12000小时人工标注评估先进RAG管道，发现视觉检索优于文本检索，晚期交互模型和文本重排序显著提升性能，但模型在非文本元素、开放性查询和细粒度视觉定位上仍有不足。
摘要：
检索增强生成（RAG）管道必须解决超越简单单文档检索的挑战，例如解释视觉元素（表格、图表、图像）、跨文档信息综合以及提供准确的来源定位。现有基准未能捕捉这种复杂性，通常专注于文本数据、单文档理解或孤立评估检索和生成。我们引入ViDoRe v3，一个综合的多模态RAG基准，涵盖10个不同专业领域的数据集，包含约26,000文档页面配以3,099个经人类验证的查询，每个查询可用6种语言。通过12,000小时的人工标注努力，我们提供了高质量的检索相关性、边界框定位和验证参考答案标注。对最先进的RAG管道评估显示，视觉检索器优于文本检索器，晚期交互模型和文本重排序显著提升性能，而混合或纯视觉上下文提高回答生成质量。然而，当前模型在非文本元素、开放性查询和细粒度视觉定位上仍有不足。为鼓励在这些挑战上的进步，该基准在商业许可下发布于https://hf.co/vidore。
英文摘要：
Retrieval-Augmented Generation (RAG) pipelines must address challenges beyond simple single-document retrieval, such as interpreting visual elements (tables, charts, images), synthesizing information across documents, and providing accurate source grounding. Existing benchmarks fail to capture this complexity, often focusing on textual data, single-document comprehension, or evaluating retrieval and generation in isolation. We introduce ViDoRe v3, a comprehensive multimodal RAG benchmark featuring multi-type queries over visually rich document corpora. It covers 10 datasets across diverse professional domains, comprising ~26,000 document pages paired with 3,099 human-verified queries, each available in 6 languages. Through 12,000 hours of human annotation effort, we provide high-quality annotations for retrieval relevance, bounding box localization, and verified reference answers. Our evaluation of state-of-the-art RAG pipelines reveals that visual retrievers outperform textual ones, late-interaction models and textual reranking substantially improve performance, and hybrid or purely visual contexts enhance answer generation quality. However, current models still struggle with non-textual elements, open-ended queries, and fine-grained visual grounding. To encourage progress in addressing these challenges, the benchmark is released under a commercially permissive license at https://hf.co/vidore.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
