arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

复杂网页任务中的结构化交互、视觉定位与鲁棒执行:WebRetriever挑战赛技术报告

Structured Interaction, Visual Localization, and Robust Execution for Complex Web Tasks: A Technical Report on the WebRetriever Challenge

Ziqi Zhang, Shaohui Li, Bing Li

arXiv 2609.35904首次发表:更新:

发表机构

Beijing Key Laboratory of Multimodal Super-Intelligent Security & People AI(北京多模态超智能安全与人民人工智能重点实验室)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本系统针对WebRetriever挑战赛,采用结构化交互优先策略,结合网格辅助视觉定位、分层上下文管理和故障感知执行机制,在协议3中取得59%通过率并获总冠军。

AI 中文摘要

本报告介绍了为WebRetriever挑战赛开发的网页智能体系统。该系统采用结构化交互优先策略,利用语义化网页信息执行常规浏览器操作,仅在结构化表示不足时调用视觉感知。系统引入了三项关键设计:用于难以访问控件的网格辅助视觉定位、用于减少冗余页面和交互历史的分层上下文管理,以及用于稳定多浏览器任务处理的故障感知执行机制。在本地评估中,系统在协议1上实现了高达79%的通过率。在官方协议3竞赛中,系统以八个并发浏览器工作线程实现了59%的通过率,并总体排名第一,赢得了WebRetriever挑战赛。

英文摘要

This report presents the web agent system developed for the WebRetriever Challenge. The system follows a structuredinteraction- first strategy, using semantic webpage information for routine browser operations and invoking visual perception only when structured representations are insufficient. Three key designs are introduced: grid-assisted visual localization for difficult-to-access controls, hierarchical context management for reducing redundant page and interaction history, and fault-aware execution mechanisms for stable multi-browser task processing. The system achieved a pass rate of up to 79% in local evaluation on Protocol 1. In the official Protocol 3 competition, it achieved a 59% pass rate with eight concurrent browser workers and ranked first overall, winning the WebRetriever Challenge.

CommentsWinning Report for the WebRetriever Challenge

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑