智能体测试智能体：用于对话式AI智能体自动化测试与评估的元智能体
Agent-Testing Agent: A Meta-Agent for Automated Testing and Evaluation of Conversational AI Agents
AI总结：
针对LLM智能体评估依赖静态基准和小规模人类研究的问题，提出结合静态代码分析等多模块、难度自适应的元智能体ATA，其在旅行规划等智能体测试中发现的故障更优且效率更高，还提供量化指标和错误报告并开源实现。
摘要：
大语言模型（LLM）智能体正越来越多地被部署用于借助工具进行规划、检索和写作，但评估仍依赖静态基准和小规模人类研究。我们提出智能体测试智能体（ATA），这是一种结合静态代码分析、设计者询问、文献挖掘和基于角色的对抗性测试生成（其难度通过评判反馈自适应调整）的元智能体。每个对话通过“以LLM为评判”（LAAJ）的评分标准进行评分，并用于引导后续测试聚焦于智能体最薄弱的能力。在旅行规划智能体和维基百科写作智能体上，ATA发现的故障比专家标注者更多样、更严重，同时故障严重程度相当；ATA仅需20-30分钟完成测试，而十名标注者的测试轮次需耗时数天。移除代码分析和网络搜索模块会增加结果方差并导致校准失准，凸显了基于证据的测试生成的价值。ATA为开发者输出定量指标和定性错误报告。我们发布完整方法学和开源实现，以实现可复现的智能体测试：https://github.com/KhalilMrini/Agent-Testing-Agent
英文摘要：
LLM agents are increasingly deployed to plan, retrieve, and write with tools, yet evaluation still leans on static benchmarks and small human studies. We present the Agent-Testing Agent (ATA), a meta-agent that combines static code analysis, designer interrogation, literature mining, and persona-driven adversarial test generation whose difficulty adapts via judge feedback. Each dialogue is scored with an LLM-as-a-Judge (LAAJ) rubric and used to steer subsequent tests toward the agent's weakest capabilities. On a travel planner and a Wikipedia writer, the ATA surfaces more diverse and severe failures than expert annotators while matching severity, and finishes in 20--30 minutes versus ten-annotator rounds that took days. Ablating code analysis and web search increases variance and miscalibration, underscoring the value of evidence-grounded test generation. The ATA outputs quantitative metrics and qualitative bug reports for developers. We release the full methodology and open-source implementation for reproducible agent testing: https://github.com/KhalilMrini/Agent-Testing-Agent
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
