arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.02626cs.AI
收藏

见与忆:一种用于网页浏览的多模态代理

See and Remember: A Multimodal Agent for Web Traversal

Shanghai Institute of AI for Education(上海人工智能教育研究院);East China Normal University(华东师范大学)

Xinjun Wang, Shengyao Wang, Aimin Zhou, Hao Hao

AI总结:

V-GEMS通过视觉 grounding 和显式记忆系统实现精确稳健的网页浏览,实验显示其在导航任务中性能提升28.7%

摘要:

自主网页导航需要代理能够感知复杂的视觉环境并维持长期上下文,但当前基于大语言模型(LLM)的代理往往在空间定向和导航循环方面遇到困难。在本文中,我们提出了通用的V-GEMS(视觉 grounding 和显式记忆系统),一种设计用于精确且稳健网页浏览的鲁棒多模态代理架构。我们的代理集成了视觉 grounding 以解决模糊的交互元素,并引入了带有状态跟踪的显式记忆堆栈。这种双重机制使代理能够维护其浏览路径的结构地图,从而实现有效的回溯并防止深度导航任务中的循环失败。我们还引入了可更新的动态基准测试以严格评估适应性。实验表明,V-GEMS在WebWalker基准上显著占优,实现了28.7%的性能提升。代码可在https://github.com/Vaultttttttttttt/V-GEMS上获得。

英文摘要:

Autonomous web navigation requires agents to perceive complex visual environments and maintain long-term context, yet current Large Language Model (LLM) based agents often struggle with spatial disorientation and navigation loops. In this paper, we propose generally applicable V-GEMS(Visual Grounding and Explicit Memory System), a robust multimodal agent architecture designed for precise and resilient web traversal. Our agent integrates visual grounding to resolve ambiguous interactive elements and introduces an explicit memory stack with state tracking. This dual mechanism allows the agent to maintain a structured map of its traversal path, enabling valid backtracking and preventing cyclical failures in deep navigation tasks. We also introduce an updatable dynamic benchmark to rigorously evaluate adaptability. Experiments show V-GEMS significantly dominates the WebWalker baseline, achieving a substantial 28.7% performance gain. Code is available at https://github.com/Vaultttttttttttt/V-GEMS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑