Wink: 代码代理中行为失误的恢复
Wink: Recovering from Misbehaviors in Coding Agents
AI总结：
Wink通过异步自我干预系统有效恢复代码代理的行为失误，减少工具调用失败和人工干预需求。
摘要：
自主编码代理，由大型语言模型（LLMs）驱动，正在越来越多地被软件行业采用，以自动化复杂的工程任务。然而，这些代理容易出现各种行为失误，例如偏离用户的指令、陷入重复的循环或无法正确使用工具。这些失败会破坏开发流程，通常需要资源密集型的手动干预。在本文中，我们提出了一种系统，用于在大规模上自动恢复代理的行为失误。我们首先介绍了一种基于生产流量分析的失误分类法，识别出三个主要类别：规范漂移、推理问题和工具调用失败，这些类别约占所有代理轨迹的30%。为了解决这些问题，我们开发了一种轻量级的异步自我干预系统，名为Wink。Wink观察代理轨迹，并提供针对性的课程修正指导，以引导代理回到 productive 的路径。我们对超过10,000个真实世界代理轨迹进行了评估，发现该系统成功解决了90%需要单次干预的失误。此外，在我们生产环境中的实时A/B测试显示，我们的系统导致工具调用失败、每会话令牌数和工程师干预次数显著减少。我们展示了设计和部署该系统的经验，提供了在大规模构建健壮代理系统方面的见解。
英文摘要：
Autonomous coding agents, powered by large language models (LLMs), are increasingly being adopted in the software industry to automate complex engineering tasks. However, these agents are prone to a wide range of misbehaviors, such as deviating from the user's instructions, getting stuck in repetitive loops, or failing to use tools correctly. These failures disrupt the development workflow and often require resource-intensive manual intervention. In this paper, we present a system for automatically recovering from agentic misbehaviors at scale. We first introduce a taxonomy of misbehaviors grounded in an analysis of production traffic, identifying three primary categories: Specification Drift, Reasoning Problems, and Tool Call Failures, which we find occur in about 30% of all agent trajectories. To address these issues, we developed a lightweight, asynchronous self-intervention system named Wink. Wink observes agent trajectories and provides targeted course-correction guidance to nudge the agent back to a productive path. We evaluated our system on over 10,000 real world agent trajectories and found that it successfully resolves 90% of the misbehaviors that require a single intervention. Furthermore, a live A/B test in our production environment demonstrated that our system leads to a statistically significant reduction in Tool Call Failures, Tokens per Session and Engineer Interventions per Session. We present our experience designing and deploying this system, offering insights into the challenges of building resilient agentic systems at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
