屏幕上的图灵测试:移动GUI代理人化的一个基准
Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization
AI总结:
本文提出屏幕上的图灵测试基准,通过建模检测器与代理的MinMax优化问题,评估代理在人类中心生态系统中的生存能力,并提出人化基准和检测指标,展示代理在不牺牲性能的前提下实现高模仿性。
摘要:
自主GUI代理的兴起引发了数字平台的对抗性反制措施,但现有研究优先考虑效用和鲁棒性,而忽视了反检测的关键维度。我们主张,为了在人类中心的生态系统中生存,代理必须发展人化能力。我们引入了“屏幕上的图灵测试”,正式将交互建模为检测器与代理之间的MinMax优化问题,旨在最小化行为差异。随后,我们收集了一个新的高保真移动触控动态数据集,并分析发现基于常规LMM的代理由于不自然的运动学而容易被检测到。因此,我们建立了代理人化基准(AHB)和检测指标,以量化模仿性与效用之间的权衡。最后,我们提出了从启发式噪声到数据驱动行为匹配的方法,证明代理可以在理论和实践中实现高模仿性,而不会牺牲性能。这项工作将范式从代理是否能完成任务,转变为在人类中心生态系统中如何完成任务,为在对抗性数字环境中实现无缝共存奠定了基础。
英文摘要:
The rise of autonomous GUI agents has triggered adversarial countermeasures from digital platforms, yet existing research prioritizes utility and robustness over the critical dimension of anti-detection. We argue that for agents to survive in human-centric ecosystems, they must evolve Humanization capabilities. We introduce the ``Turing Test on Screen,'' formally modeling the interaction as a MinMax optimization problem between a detector and an agent aiming to minimize behavioral divergence. We then collect a new high-fidelity dataset of mobile touch dynamics, and conduct our analysis that vanilla LMM-based agents are easily detectable due to unnatural kinematics. Consequently, we establish the Agent Humanization Benchmark (AHB) and detection metrics to quantify the trade-off between imitability and utility. Finally, we propose methods ranging from heuristic noise to data-driven behavioral matching, demonstrating that agents can achieve high imitability theoretically and empirically without sacrificing performance. This work shifts the paradigm from whether an agent can perform a task to how it performs it within a human-centric ecosystem, laying the groundwork for seamless coexistence in adversarial digital environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。