arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.02795cs.CV
收藏

VSearcher:通过强化学习实现长周期多模态搜索代理

VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learning

University of Macau(澳门大学)

Ruiyang Zhang, Qianguo Sun, Chao Song, Yiyan Qi, Zhedong Zheng

AI总结:

VSearcher通过强化学习实现多模态搜索代理,提升长周期多轮工具使用能力,在多模态网络搜索任务中表现优异。

摘要:

大型模型正日益成为能够与现实世界环境互动并利用外部工具来增强其静态能力的自主代理。然而,大多数最近的进步都集中在纯文本大语言模型上,这些模型仅限于单一模态,因此应用场景较为狭窄。另一方面,多模态大模型虽然提供了更强的感知能力,但仍局限于静态知识,并缺乏获取和利用最新网络信息的能力。在本文中,我们提出了VSearcher,将静态多模态模型转变为能够实现在现实网络环境中进行长周期、多轮工具使用的多模态搜索代理,包括文本搜索、图像搜索和网页浏览。具体来说,我们引入了迭代注入数据合成流程来生成大规模、复杂的多模态问答问题,这些问题随后通过综合指标进行过滤,以确保高质量和足够的难度。然后,我们采用SFT-然后-RL训练流程,将基础多模态模型转变为能够在现实网络环境中进行多轮工具调用的代理。此外,我们提出了一个专门用于评估多模态搜索代理搜索能力的多模态搜索基准MM-SearchExam,这证明了最近的专有模型在该任务上具有高度挑战性。在多个多模态搜索基准上的广泛评估揭示了我们方法的有效性。VSearcher在多个多模态网络搜索任务中优于最近的多模态搜索代理,甚至在某些多模态网络搜索任务中超过了几个专有模型。

英文摘要:

Large models are increasingly becoming autonomous agents that interact with real-world environments and use external tools to augment their static capabilities. However, most recent progress has focused on text-only large language models, which are limited to a single modality and therefore have narrower application scenarios. On the other hand, multimodal large models, while offering stronger perceptual capabilities, remain limited to static knowledge and lack the ability to access and leverage up-to-date web information. In this paper, we propose VSearcher, turning static multimodal model into multimodal search agent capable of long-horizon, multi-turn tool use in real-world web environments, including text search, image search, and web browsing, via reinforcement learning. Specifically, we introduce Iterative Injection Data Synthesis pipeline to generate large-scale, complex multimodal QA questions, which are further filtered with comprehensive metrics to ensure high quality and sufficient difficulty. We then adopt an SFT-then-RL training pipeline to turn base multimodal models to agent capable of multi-turn tool calling in real-world web environments. Besides, we propose a multimodal search benchmark MM-SearchExam dedicated to evaluating search capabilities of multimodal search agents, which proves highly challenging for recent proprietary models. Extensive evaluations across multiple multimodal search benchmarks reveal effectiveness of our method. VSearcher achieves superior performance compared to recent multimodal search agents and even surpasses several proprietary models on multimodal web search tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑