发表机构
KAIST; Yale; University of Haifa; UCL; UvA; SNU（韩国科学技术院; 耶鲁大学; 海法大学; 伦敦大学学院; 阿姆斯特丹大学; 首尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究人员开发AEROBAT多智能体系统，自动化AI智能体行为科学研究流程，经实验验证其可生成测试假设、开展大量对照实验与模拟，能补充拓展人工研究。
AI 中文摘要
随着AI智能体在复杂环境中部署日益广泛，理解其行为变得至关重要。然而针对AI智能体的行为科学研究仍停留在人工且劳动密集的阶段。我们推出AEROBAT，首个用于自动化AI智能体行为科学研究的多智能体系统。针对用户指定的任意目标行为，AEROBAT可自动执行完整的行为科学研究流程：生成关于该行为的假设、设计并开展对照实验、进行行为评估、分析结果以及撰写报告。针对12种目标行为，我们借助AEROBAT生成并测试了79个假设，累计设计1240项对照实验并完成23512轮模拟。研究发现26个假设存在中等到强的统计证据，其中部分为新假设。综上，我们的结果表明，针对AI智能体的自动化行为科学研究可补充并拓展人工研究的覆盖范围。
英文摘要
As AI agents are increasingly deployed in complex and new environments, knowing the conditions that influence their behavior becomes an indispensable step for their reliable and safe deployment. Yet causal behavioral evaluation of AI agents remains manual and labor-intensive. We introduce Abs2Sim and AEROBAT, a system of methods that support causal behavioral evaluation of AI agents via automated behavioral science. Given a user-specified target behavior, the methods automatically execute a full pipeline of behavioral science research---generating hypotheses about the behavior, designing and executing controlled experiments, making behavioral assessments, analyzing the results, and writing reports. For 12 target behaviors, the methods generated and tested 73 hypotheses: designing 1,160 controlled experiments and executing 22,954 simulation rounds in total. Moderate-to-strong statistical evidence emerged for 30 hypotheses, revealing potential modulators of AI behavior. In sum, our results demonstrate that automated behavioral science can extend the reach of behavioral evaluation of AI agents.
Commentspreprint
