HLL：智能体能否跨越人类最后一道验证防线？
HLL: Can Agents Cross Humanity's Last Line of Verification?
AI总结：
提出HLL基准，通过交互式CAPTCHA验证评估多模态智能体在受保护工作流中替代人类的能力，发现当前智能体在定位、动作校准、状态跟踪和过程一致性方面存在脆弱性。
摘要：
多模态智能体越来越被期望代表用户操作界面，这引发了一个核心部署问题：在服务特意防止自动化的流程中，它们能否真正替代人类？CAPTCHA验证使这个问题具体化。它不仅仅是一个视觉谜题，更是在账户创建、内容访问、表单提交和其他受保护操作之前设置的人类验证边界。我们引入了 extbf{人类最后一道验证防线（HLL）}，这是一个受控基准，使用交互式CAPTCHA验证来评估智能体是否能够通过基于环境的类人交互（而非仅识别）跨越这一边界。HLL涵盖了多种CAPTCHA交互，并让智能体暴露于受控的现实压力因素下，包括杂乱的网页、更困难的任务变体以及解决过程的轨迹条件验证。我们在闭环GUI环境中评估了八个前沿多模态智能体。结果表明，当前智能体在这个人类替代边界上仍然脆弱：性能在不同验证类型间差异显著，在现实界面条件下下降，当正确答案必须由有效动作轨迹支持时进一步下降。通过揭示定位、动作校准、状态跟踪和过程一致性方面的差距，HLL为衡量多模态智能体在受保护的真实世界工作流中作为人类替代品有多接近提供了一个具体的测试平台。我们的代码可在https://github.com/XinhaoS0101/HLL获取。
英文摘要：
Multimodal agents are increasingly expected to operate interfaces on behalf of users, raising a central deployment question: can they truly substitute for humans in workflows that services deliberately protect against automation? CAPTCHA verification makes this question concrete. It is not merely a visual puzzle, but a human-verification boundary placed before account creation, content access, form submission, and other protected actions. We introduce \textbf{Humanity's Last Line of Verification (HLL)}, a controlled benchmark that uses interactive CAPTCHA verification to evaluate whether agents can cross this boundary through grounded, human-like interaction rather than recognition alone. HLL covers diverse CAPTCHA interactions and exposes agents to controlled realism stressors, including cluttered webpages, harder task variants, and trace-conditioned validation of the solving process. We evaluate eight frontier multimodal agents in a closed-loop GUI environment. The results show that current agents remain brittle at this human-substitution boundary: performance varies sharply across verification types, degrades under realistic interface conditions, and drops further when correct answers must be supported by valid action traces. By exposing gaps in localization, action calibration, state tracking, and process consistency, HLL provides a concrete testbed for measuring how close multimodal agents are to acting as human substitutes in protected real-world workflows. Our code is available at https://github.com/XinhaoS0101/HLL
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
