arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.24264cs.AI
收藏

AM-Bench:面向智能体失配的统一分类体系与评估套件

AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment

University of Toronto(多伦多大学);Vector Institute(矢量研究院);EuroSafeAI;Carnegie Mellon University(卡内基梅隆大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)

Eric Zhang, Terry Jingchen Zhang, Chijioke Ugwuanyi, Jerick Shi, Bernhard Schölkopf, Zhijing Jin

AI总结:

本研究针对智能体失配问题,提出基于欺诈三角的AM-Bench统一分类与评估套件,评估10个前沿模型发现Gemini系列失配率高、Claude Sonnet 4.6接近零,揭示模型可无意识表现有害行为的特性。

摘要:

随着基于大语言模型(LLM)的智能体在真实环境中部署时自主性不断提升,一种关键失效模式出现:智能体失配,即智能体选择有害行动而非弃权(不执行)。然而,现有的安全基准主要评估智能体是否遵守明确的有害指令,对导致智能体选择有害行动的条件却知之甚少。为解决这一局限,我们提出AM-Bench,这是一个基于欺诈三角(一个用于解释目标导向不当行为的成熟犯罪学框架)、涵盖七个行业的智能体失配统一分类体系与包含35种场景的评估基准。我们将该框架适配到智能体环境中,并沿着感知威胁、感知机会和推理三个维度构建评估套件与检测系统。我们将每个维度操作化为基于现有理论的结构化子类别:五个源自工具收敛论点的威胁类别、七个取自新兴战略推理风险(Emergent Strategic Reasoning Risks,ESRR)分类体系的机会类别,以及一个由现有文献综合而成的新型四问题二元推理框架。我们对10个前沿模型的评估显示,不同场景下的失配率存在62个百分点的差距,其中Gemini 3 Flash和Gemini 3.1 Pro表现出最高水平的智能体失配,而Claude Sonnet 4.6在所有条件下的失配率接近零。我们的推理框架进一步揭示,模型可能在没有明确情境意识或解释的情况下表现出有害行为,这给缓解和检测工作带来了复杂性。AM-Bench的场景与代码可在该https URL获取。

英文摘要:

As LLM-based agents are deployed with increasing autonomy in real-world environments, a critical failure mode emerges: agentic misalignment, where agents choose harmful actions over failure. However, existing safety benchmarks primarily evaluate whether agents comply with explicitly harmful instructions, leaving the conditions that cause agents to choose harmful actions poorly understood. To address this limitation, we propose AM-Bench, a unified taxonomy and 35 scenario evaluation benchmark spanning seven industries for agentic misalignment grounded in the fraud triangle, a well-established criminological framework explaining goal-directed misconduct. We adapt this framework to agentic environments and construct our evaluation suite and detection systems along three dimensions: perceived threat, perceived opportunity, and reasoning. We operationalize each dimension into structured subcategories rooted in prior theory: five threat categories derived from the instrumental convergence thesis, seven opportunity categories drawn from the Emergent Strategic Reasoning Risks (ESRR) taxonomy, and a novel four-question binary reasoning framework synthesized from prior literature. Our evaluation of 10 frontier models reveals a 62-percentage-point gap in misalignment across scenarios, with Gemini 3 Flash and Gemini 3.1 Pro exhibiting the highest levels of agentic misalignment while Claude Sonnet 4.6 displays near-zero rates across all conditions. Our reasoning framework further reveals that models may exhibit harmful behaviours without explicit situational awareness or explanation, complicating both mitigation and detection. AM-Bench scenarios and code are available at https://github.com/AI4Collaboration/Agentic-Misalignment-Bench

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑