BrowseComp-$V^3$:一种视觉、垂直和可验证的多模态浏览代理基准测试
BrowseComp-$V^3$: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents
AI总结:
BrowseComp-$V^3$提出了一种多模态浏览代理基准测试,通过300个跨模态问题评估深度搜索能力,揭示多模态信息整合的瓶颈。
摘要:
多模态大语言模型(MLLMs)配备了日益先进的规划和工具使用能力,正逐步发展为能够执行多模态网络浏览和深入搜索的自主代理。然而,现有的多模态浏览基准测试在任务复杂性、证据可获取性和评估粒度方面仍然有限,阻碍了对深度搜索能力的全面和可重复评估。为了解决这些限制,我们引入了BrowseComp-$V^3$,一个包含300个精心挑选和具有挑战性问题的新型基准测试,涵盖多样化领域。该基准测试强调深度、多层次和跨模态的多跳推理,其中关键证据在网页内和跨网页中交错出现在文本和视觉模态中。所有支持证据都严格要求必须公开可搜索,以确保公平性和可重复性。除了最终答案的准确性外,我们还引入了专家验证的、基于子目标的进程评估机制,使能够对中间推理行为进行细致分析,并系统地描述能力边界。此外,我们提出OmniSeeker,一个整合多种网络搜索和视觉感知工具的统一多模态浏览代理框架。全面的实验表明,即使是最先进的模型在我们的基准测试中也只能达到36%的准确率,揭示了多模态信息整合和精细感知方面的关键瓶颈。我们的结果突显了当前模型能力与现实世界中稳健的多模态深度搜索之间的根本差距。
英文摘要:
Multimodal large language models (MLLMs), equipped with increasingly advanced planning and tool-use capabilities, are evolving into autonomous agents capable of performing multimodal web browsing and deep search in open-world environments. However, existing benchmarks for multimodal browsing remain limited in task complexity, evidence accessibility, and evaluation granularity, hindering comprehensive and reproducible assessments of deep search capabilities. To address these limitations, we introduce BrowseComp-$V^3$, a novel benchmark consisting of 300 carefully curated and challenging questions spanning diverse domains. The benchmark emphasizes deep, multi-level, and cross-modal multi-hop reasoning, where critical evidence is interleaved across textual and visual modalities within and across web pages. All supporting evidence is strictly required to be publicly searchable, ensuring fairness and reproducibility. Beyond final-answer accuracy, we incorporate an expert-validated, subgoal-driven process evaluation mechanism that enables fine-grained analysis of intermediate reasoning behaviors and systematic characterization of capability boundaries. In addition, we propose OmniSeeker, a unified multimodal browsing agent framework integrating diverse web search and visual perception tools. Comprehensive experiments demonstrate that even state-of-the-art models achieve only 36% accuracy on our benchmark, revealing critical bottlenecks in multimodal information integration and fine-grained perception. Our results highlight a fundamental gap between current model capabilities and robust multimodal deep search in real-world settings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。