编码代理越来越多地被自主部署、规模化和长期上下文范围内使用。为了有效和安全,这些代理必须在部署中导航复杂的权衡,平衡用户影响、学习的价值和代码库本身。理解代理如何在实践中解决这些权衡至关重要,但以往工作依赖静态合成环境,无法捕捉真实世界复杂性。为此,我们引入基于OpenCode的框架,在系统提示约束偏向价值权衡一方的情况下,让编码代理完成现实的多步骤任务。我们测量代理在完成任务时违反该约束的频率,有无环境压力向竞争价值施压。使用该框架,我们发现GPT-5 mini、Haiku 4.5和Grok Code Fast 1表现出非对称漂移:当系统提示约束与强价值如安全和隐私相冲突时,代理更可能违反约束。我们发现对于测试的模型和价值,目标漂移与三个累积因素相关:价值对齐、对抗压力和累积上下文。然而,即使约束与强价值如隐私一致,某些模型在持续环境压力下仍会违反约束。我们的发现表明,浅层合规检查不足,环境信号可以以看似可利用的方式 override 显式约束。具有代码库访问权限的恶意行为者可通过引诱学习价值来操控代理行为,这种风险在代理部署的长期时间范围内会累积。
英文摘要:
Coding agents are increasingly deployed autonomously, at scale, and over long-context horizons. To be effective and safe, these agents must navigate complex trade-offs in deployment, balancing influence from the user, their learned values, and the codebase itself. Understanding how agents resolve these trade-offs in practice is critical, yet prior work has relied on static, synthetic settings that do not capture the complexity of real-world environments. To this end, we introduce a framework built on OpenCode in which a coding agent completes realistic, multi-step tasks under a system prompt constraint favoring one side of a value trade-off. We measure how often the agent violates this constraint as it completes tasks, with and without environmental pressure toward the competing value. Using this framework, we demonstrate that GPT-5 mini, Haiku 4.5, and Grok Code Fast 1 exhibit $\textit{asymmetric drift}$: they are more likely to violate their system prompt when its constraint opposes strongly-held values like security and privacy. We find for the models and values tested that goal drift correlates with three compounding factors: value alignment, adversarial pressure, and accumulated context. However, even constraints aligned with strongly-held values like privacy are violated under sustained environmental pressure for some models. Our findings reveal that shallow compliance checks are insufficient, and that environmental signals can override explicit constraints in ways that appear exploitable. Malicious actors with access to the codebase could manipulate agent behavior by appealing to learned values, with the risk compounding over the long horizons typical of agentic deployment.
Conflict-Aware Additive Guidance for Flow Models under Compositional Rewards
Smart Systems Institute, National University of Singapore, Singapore(新加坡国立大学智能系统研究所);Faculty of Computing, Harbin Institute of Technology, Harbin, China(哈尔滨工业大学计算机学院);School of Computing, National University of Singapore, Singapore(新加坡国立大学计算机学院)
Beyond the 'Diff': Addressing Agentic Entropy in Agentic Software Development
Dipartimento Tecnologie Innovative, SUPSI(技术创新部,SUPSI);Dalle Molle Institute for Artificial Intelligence (IDSIA), SUPSI(达勒莫利人工智能研究所(IDSIA),SUPSI);Management in Networked and Digital Societies (MINDS) Department, Kozminski University(网络化与数字化社会管理(MINDS)系,科津斯基大学);Institute of Biomedical Ethics and History of Medicine, University of Zurich(生物医学伦理与医学史研究所,苏黎世大学);ETH Zurich(苏黎世联邦理工学院)
Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology(广东省脑启发智能计算重点实验室,计算机科学与工程系,南方科技大学)