arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2604.09574cs.AIcs.LG
收藏

屏幕上的图灵测试:移动GUI代理人化的一个基准

Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization

Shanghai Jiao Tong University(上海交通大学);Carnegie Mellon University(卡内基梅隆大学)

Jiachen Zhu, Lingyu Yang, Rong Shan, Congmin Zheng, Zeyu Zheng, Weiwen Liu, Yong Yu, Weinan Zhang, Jianghao Lin

AI总结:

本文提出屏幕上的图灵测试基准,通过建模检测器与代理的MinMax优化问题,评估代理在人类中心生态系统中的生存能力,并提出人化基准和检测指标,展示代理在不牺牲性能的前提下实现高模仿性。

摘要:

自主GUI代理的兴起引发了数字平台的对抗性反制措施,但现有研究优先考虑效用和鲁棒性,而忽视了反检测的关键维度。我们主张,为了在人类中心的生态系统中生存,代理必须发展人化能力。我们引入了“屏幕上的图灵测试”,正式将交互建模为检测器与代理之间的MinMax优化问题,旨在最小化行为差异。随后,我们收集了一个新的高保真移动触控动态数据集,并分析发现基于常规LMM的代理由于不自然的运动学而容易被检测到。因此,我们建立了代理人化基准(AHB)和检测指标,以量化模仿性与效用之间的权衡。最后,我们提出了从启发式噪声到数据驱动行为匹配的方法,证明代理可以在理论和实践中实现高模仿性,而不会牺牲性能。这项工作将范式从代理是否能完成任务,转变为在人类中心生态系统中如何完成任务,为在对抗性数字环境中实现无缝共存奠定了基础。

英文摘要:

The rise of autonomous GUI agents has triggered adversarial countermeasures from digital platforms, yet existing research prioritizes utility and robustness over the critical dimension of anti-detection. We argue that for agents to survive in human-centric ecosystems, they must evolve Humanization capabilities. We introduce the ``Turing Test on Screen,'' formally modeling the interaction as a MinMax optimization problem between a detector and an agent aiming to minimize behavioral divergence. We then collect a new high-fidelity dataset of mobile touch dynamics, and conduct our analysis that vanilla LMM-based agents are easily detectable due to unnatural kinematics. Consequently, we establish the Agent Humanization Benchmark (AHB) and detection metrics to quantify the trade-off between imitability and utility. Finally, we propose methods ranging from heuristic noise to data-driven behavioral matching, demonstrating that agents can achieve high imitability theoretically and empirically without sacrificing performance. This work shifts the paradigm from whether an agent can perform a task to how it performs it within a human-centric ecosystem, laying the groundwork for seamless coexistence in adversarial digital environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑