arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.04582cs.AIcs.LG
收藏

自我归因偏差:当AI监控对自己宽容时

Self-Attribution Bias: When AI Monitors Go Easy on Themselves

Dipika Khullar, Jack Hopkins, Rowan Wang, Fabien Roger

AI总结:

研究发现AI监控在自我归因偏差下可能低估风险,导致部署不充分的监控系统。

摘要:

代理系统越来越多地依赖语言模型来监控自身行为。例如,编码代理可能会自我批评生成的代码以批准拉取请求,或评估工具使用行为的安全性。我们证明,当行为出现在前一个或同一助手回合中而非由用户在用户回合中呈现时,这种设计模式可能会失效。我们将自我归因偏差定义为模型倾向于将行为视为更正确或更少风险,当该行为被隐含地框架为自身的行为,与当该行为在非策略性归因下评估时相比。在四个编码和工具使用数据集中,我们发现,当评估跟随一个先前助手回合中生成该行为时,监控器更少报告高风险或低正确性行为,与当相同行为在新上下文中由用户回合呈现时相比。相反,明确指出行为来自监控器本身并不会本身引发自我归因偏差。因为监控器通常基于固定示例而非自身生成的行为进行评估,这些评估可能导致监控器在部署中显得比实际更可靠,导致开发者无意中部署了不充分的监控器。

英文摘要:

Agentic systems increasingly rely on language models to monitor their own behavior. For example, coding agents may self critique generated code for pull request approval or assess the safety of tool-use actions. We show that this design pattern can fail when the action is presented in a previous or in the same assistant turn instead of being presented by the user in a user turn. We define self-attribution bias as the tendency of a model to evaluate an action as more correct or less risky when the action is implicitly framed as its own, compared to when the same action is evaluated under off-policy attribution. Across four coding and tool-use datasets, we find that monitors fail to report high-risk or low-correctness actions more often when evaluation follows a previous assistant turn in which the action was generated, compared to when the same action is evaluated in a new context presented in a user turn. In contrast, explicitly stating that the action comes from the monitor does not by itself induce self-attribution bias. Because monitors are often evaluated on fixed examples rather than on their own generated actions, these evaluations can make monitors appear more reliable than they actually are in deployment, leading developers to unknowingly deploy inadequate monitors in agentic systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑