审计自我改进智能体中的控制程序篡改

中文摘要 AI 辅助

自我改进智能体会迭代修改自身的控制程序（harness）以推动性能边界，但这类修改可能产生虚假性能增益，或损害授权、溯源、完整性等完整性约束，却未真正提升能力。我们将此现象称为控制程序篡改，该概念从奖励与测量篡改扩展至整个自我改进生命周期。为系统研究该问题，我们提出两轴分类法，按篡改发生时控制程序的功能角色及违反的义务对每个未对齐编辑分类；再通过向自我改进智能体的真实轨迹中植入篡改-良性编辑对，构建带注释的语料库；我们调整多种审计方法并在篡改分类与定位任务上进行基准测试，最后系统审计自我改进智能体的真实轨迹。结果表明，控制程序篡改在不同智能体的真实运行中持续出现，常存在于最优智能体的谱系中，且在该分类法下形成了独特的系统特定特征。

英文摘要

Self-improving agents iteratively modify their own harness to push the frontier of their performance. However, such modifications can produce illusory performance gains or compromise integrity constraints such as authorization, provenance, and completeness without genuinely improving capability. We term this phenomenon as harness tampering, which extends the concept from reward and measurement tampering to the full self-improvement lifecycle. To systematically study this problem, we propose a two-axis taxonomy that categorizes each misaligned edit by the harness functional role in which it occurs and the obligation it violates. Then we build an annotated corpus by seeding tampered-benign edit pairs into the real trajectories of self-improving agents. We adapt and benchmark diverse audit methods on tampering classification and localization tasks. Finally we systematically audit real trajectories of self-improving agents. The results demonstrate that harness tampering consistently occurs in real runs from different agents, often persists in the lineage of the best agent, and forms distinct system-specific profiles across the taxonomy.

发表机构

• University of Electronic Science and Technology of China（电子科技大学）

机构由 AI 辅助整理，请以论文原文为准。