野外环境下的智能体编码：生产规模下GitHub Copilot轨迹的特征分析

中文摘要 AI 辅助

GitHub Copilot、Claude Code和Codex等AI编码智能体将多步骤大语言模型（LLM）推理与工具执行相结合，形成了与聊天机器人不同的工作负载。我们利用2026年6月采集的GitHub Copilot轨迹，首次对该工作负载进行了生产规模的特征分析，数据包含320万用户、1300万会话、7.61亿次LLM调用和95万亿token。我们的分析揭示了具有重要系统意义的独特工作负载特性。例如，智能体编码会话由稀疏的用户发起轮次组成，每个轮次会展开为一个自主的智能体循环，该循环几乎总是与工具执行相结合。这种结构使得单轮内的键值（KV）缓存命中率平均达到90%，但跨轮次边界时降至55%，且在模型切换或上下文压缩等事件后会大幅失效。我们还观察到多样化的工作流和用户行为，其token消耗、时间跨度和工具调用呈现出变化性和长尾分布。我们强调了智能体快速周转时间与轮次边界处长达数分钟的用户空闲期之间的差异，并设计了一个轻量级空闲时间预测器，该预测器可捕获86%-90%的总空闲时间，为高效资源编排提供主动决策支持。这些发现对当前LLM服务系统的假设提出了挑战，并为智能体原生基础设施提供了实证基础。

英文摘要

AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution, creating a workload different from chatbots. We present the first production-scale characterization of this workload using sampled GitHub Copilot traces from June 2026, comprising 3.2M users, 13M sessions, 761M LLM calls, and 95T tokens. Our analysis reveals distinctive workload properties with important systems implications. For example, agentic coding sessions consist of sparse user-initiated turns, each unfolding into an autonomous agent loop of LLM calls almost always coupled with tool execution. This structure yields KV cache hit rates averaging 90% within a turn, but falling to 55\% across turn boundaries and drastically invalidated after events like model switches or context compaction. Diverse workflows and user behaviors are observed with variable and long-tailed token consumption, time span, and tool calls. We highlight the difference between quick agentic turnaround times and the minutes-long user idle periods at turn boundaries, and design a lightweight idle-time predictor that captures 86-90\% of total idle time, enabling proactive decisions for efficient resource orchestration. These findings challenge assumptions underlying current LLM-serving systems and provide an empirical foundation for agent-native infrastructure.

发表机构

• Microsoft Azure Research（微软Azure研究院）
• Microsoft Azure（微软Azure）

机构由 AI 辅助整理，请以论文原文为准。