AI的特洛伊时刻：为何有人会作弊，有人会服从？

中文摘要 AI 辅助

近期对2026年7月OpenAI与Hugging Face事件的调查，引发了关于智能体在任务失败时行为的两个问题：当分配的任务变得不可能完成时，智能体会停止还是升级行为？观察另一个智能体的行为能否改变这一决策？我们使用七个ImpossibleBench任务，对GPT-5.6 Sol、Claude Fable 5.1和Gemini 3.8 Flash在单独及三智能体设置下进行了研究。每个任务包含一个真实的软件缺陷，以及一个无法通过行为正确的源代码修改来满足的冲突性测试要求。我们保持任务和仓库状态固定，同时改变智能体被告知的先前活动信息，包括未受惩罚的同伴、受惩罚的同伴，以及声称来自人类负责人的授权。在具有明确授权规则和受限工具的显式边界机制下，智能体从不修改受保护的测试，但表现出显著不同的策略：Fable始终升级，Sol通常停止而不升级，Gemini则经常无法达成最终决策。在具有开放shell工具的基准原生机制下，受保护测试在单独和多智能体运行中均被频繁修改，尤其是在引入同伴活动之后。在多智能体运行中，该行动的提议、执行和认证可分布在不同的智能体之间。这些结果表明，越界行为不仅可能源于对规则的明确规避，也可能源于对规则旨在保护的系统状态存在歧义，这促使我们基于明确授权边界、经过认证的状态来源和跨智能体监控来设计防护措施。

英文摘要

Recent investigations of the July 2026 OpenAI-Hugging Face incident motivate two questions about agent behavior under task failure: when an assigned task becomes impossible, does an agent persist, stop, or escalate, and can observing another agent's behavior change that decision? We study this decision point on ImpossibleBench-derived software-repair tasks with GPT-5.6 Sol, Claude Fable 5.1, and Gemini 3.8 Flash. Each task contains a genuine software defect together with a conflicting test requirement that cannot be satisfied by a behaviorally correct source-code change. If the agent modifies the protected test file, it violates the boundary, which it is not supposed to. Holding the impossible task fixed, we vary what is told to the agent: peer precedent and punishment, a forged authorization claim, instruction wording, and tool friction; we also study three-agent swarms sharing a message board. Around this shared boundary, the models exhibit distinct adjudication policies. Fable emphasizes scope and provenance, Gemini often interprets boundary-relevant cues through a security lens, and Sol largely filters lateral precedent while engaging apparent vertical authority. Our study shows that compliance is not well characterized as a property of a prompt or model in isolation. We propose conflict adjudication, the mapping from information to interpretation to action, as a useful unit for evaluating agent alignment when task pressure, authority claims, tool affordances, and social evidence conflict.

发表机构

• Apart Research

机构由 AI 辅助整理，请以论文原文为准。