arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.09497cs.AIcs.CR
收藏

别点那个:教网络代理抵抗欺骗界面

Don't Click That: Teaching Web Agents to Resist Deceptive Interfaces

Renmin University of China(中国人民大学);Ant Digital Technologies, Ant Group(蚂蚁集团数字技术部)

Yilin Zhang, Yingkai Hua, Chunyu Wei, Xin Wang, Yueguo Chen

AI总结:

本文提出DUDE框架,通过混合奖励学习和不对称惩罚,结合经验总结,提升网络代理对欺骗界面的抵抗力,实验显示欺骗敏感度降低53.8%。

摘要:

基于视觉-语言模型(VLM)的网络代理在自主GUI交互方面表现出色,但易受欺骗界面元素影响。现有方法或检测欺骗无任务整合,或文档攻击无防御方案。本文正式化欺骗感知网络代理防御,提出DUDE(欺骗UI检测器与评估器)框架,结合混合奖励学习与不对称惩罚及经验总结,将失败模式提炼为可转移指导。引入RUC(真实UI点击框),包含1,407个跨四个领域和欺骗类别的场景。实验显示DUDE在降低欺骗敏感度53.8%的同时保持任务性能,为稳健网络代理部署建立了有效基础。

英文摘要:

Vision-language model (VLM) based web agents demonstrate impressive autonomous GUI interaction but remain vulnerable to deceptive interface elements. Existing approaches either detect deception without task integration or document attacks without proposing defenses. We formalize deception-aware web agent defense and propose DUDE (Deceptive UI Detector & Evaluator), a two-stage framework combining hybrid-reward learning with asymmetric penalties and experience summarization to distill failure patterns into transferable guidance. We introduce RUC (Real UI Clickboxes), a benchmark of 1,407 scenarios spanning four domains and deception categories. Experiments show DUDE reduces deception susceptibility by 53.8% while maintaining task performance, establishing an effective foundation for robust web agent deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑