在线调查中智能体AI能力与数据质量控制的竞赛
The Race between Agentic AI Capabilities and Data Quality Control in Online Surveys
AI总结：
该研究针对在线调查的注意力检查防线，探究智能体AI完成调查并通过检查的能力，分析其结构漏洞并提出DOM元数据混淆的防御策略，评估相关模型以平衡不同研究者的需求。
AI中文摘要：
在线调查是各领域基础的数据收集工具，注意力检查是保障回答质量的关键防线。但智能体AI（由大语言模型（LLM）驱动的目标导向系统和/或具备工具增强能力的多模态处理单元）的快速兴起，对这些防护措施的稳健性提出了新问题。本研究探究智能体AI架构完成在线调查并通过标准注意力检查的能力，在受控调查沙箱中评估具备多模态输入处理和基于工具的网页交互能力的单智能体架构，从两个视角分析问题：攻击视角下，暴露的DOM元数据、可预测的选项编码等结构漏洞，使智能体仅通过结构化解析就能通过注意力检查；防御视角下，实施DOM元数据混淆的缓解策略以消除文本问题中的语义线索。评估多个开源语言和多模态模型以研究能力与协同效果，基于评估结果，为同时满足实证研究者和智能体AI研究者的需求提供见解。
英文摘要：
Online surveys are a foundational data collection instrument in a variety of fields, with attention checks serving as critical guardians of response quality. However, the rapid emergence of agentic AI (goal directed systems powered by a large language model (LLM) brain and/or a multimodal processing unit with tool-augmented capabilities) raises new questions about the robustness of these safeguards. We investigate how well agentic AI architectures can complete web-based surveys and pass standard attention checks. We evaluate a single-agent architecture capable of multimodal input processing and tool-based web interaction on a controlled survey sandbox. We analyze the problem from two perspectives. From an attack perspective, we demonstrate how structural vulnerabilities such as exposed DOM metadata and predictable option encoding allow agents to resolve attention checks through structured parsing only. From a defense perspective, we implement a mitigation strategy of DOM metadata obfuscation to remove semantic cues in text-based questions. We evaluate multiple open-source language and multimodal models to study capability and orchestration effectiveness. Based on our evaluations, we offer perspectives on how to simultaneously meet the needs of empiricists and agentic AI researchers.
