发表机构
UC Berkeley; Meta Superintelligence Labs; University of Washington; Princeton University（加州大学伯克利分校; Meta超级智能实验室; 华盛顿大学; 普林斯顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出智能体可塑性概念，衡量智能体将经验转化为未来保留性能提升的效率，并通过受控实验揭示前沿模型改进轨迹差异及失败瓶颈。
AI 中文摘要
人工智能智能体越来越多地运行在能够诊断失败并通过经验改进的环境中，然而现有的评估大多衡量智能体在某一固定时间点能做什么，而非其学习的有效性。评估自我改进需要回答三个问题：未来性能是否提升并泛化到促成学习的交互之外；新能力的获取效率如何；以及自我改进过程在何处失效？为回答这些问题，我们在一个受控环境中研究自我改进，其中智能体将过去的经验摊销为可复用的工件，并由未来的实例继承。在每个检查点，我们衡量在训练和保留的环境交互上的性能，同时考虑学习成本。我们引入了智能体可塑性，即智能体将经验转化为未来保留性能提升的效率。在多个环境中，前沿模型尽管有可比较的学习机会，却表现出截然不同的改进轨迹。一些模型取得了显著且持久的提升，而另一些则保持在初始性能附近或以下，且训练范围内的提升往往仅部分转移到分布外条件。最终能力与获取效率也存在分歧：最终表现最佳的智能体未必是改进最有效的那个。通过改进循环追溯失败，进一步揭示了不同的候选瓶颈。可塑性低的智能体往往未能复用相关工件，而可塑性较高的智能体即使复用了相关工件仍可能失败，指向工件质量、泛化或应用方面的局限。评估自我改进的智能体不仅需要衡量它们能做什么，还需要衡量它们通过经验变得更好的有效性。
英文摘要
AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns. Evaluating self-improvement requires answering three questions: does future performance improve and generalize beyond the interactions that enabled learning; how efficiently are new capabilities acquired; and where does the self-improvement process break down? To answer these questions, we study self-improvement in a controlled setting where agents amortize past experience into reusable artifacts that are inherited by future instances. At each checkpoint, we measure performance on training and held-out environment interactions while accounting for learning cost. We introduce agent plasticity, the efficiency with which an agent converts experience into gains in future held-out performance. Across multiple environments, frontier models exhibit sharply different improvement trajectories despite comparable opportunities to learn. Some achieve substantial and persistent gains, while others remain near or below their initial performance, and gains within the training regime often transfer only partially to out-of-distribution conditions. Endpoint capability and acquisition efficiency also diverge: the agent that ultimately performs best need not be the one that improves most efficiently. Tracing failures through the improvement loop further reveals different candidate bottlenecks. Agents with low plasticity often fail to reuse relevant artifacts, whereas more plastic agents may still fail despite reusing relevant artifacts, pointing to limitations in artifact quality, generalization, or application. Evaluating self-improving agents requires measuring not only what they can do, but how effectively they become better through experience.
