arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.01418cs.SEcs.AIcs.HC
收藏

命令行AI编码代理的采用与影响:微软2026年初推出Claude Code和GitHub Copilot CLI的研究

Adoption and Impact of Command-Line AI Coding Agents: A Study of Microsoft's Early 2026 Rollout of Claude Code and GitHub Copilot CLI

Microsoft(微软)

Emerson Murphy-Hill, Jenna Butler, Alexandra Savelieva

AI总结:

研究微软2026年初推出Claude Code和GitHub Copilot CLI时,数万名工程师的采用模式、留存因素及对代码合并的影响,发现社交网络驱动首次使用,编码活跃度关联留存,采用者合并PR数量增加约24%。

摘要:

组织在推广像Anthropic的Claude Code和GitHub的Copilot CLI这样的代理型命令行工具时,需要知道谁会尝试、谁会持续使用,以及这些工具是否能产生足够的输出来证明其成本。在组织规模下,token支出每年可能达到数百万美元,因此对采用、留存或影响的误判可能导致推广成本高昂而无法改变工程速度。通过研究微软2026年初推广期间数万名工程师,我们发现首次使用主要通过社交网络传播,留存与工程师的编码活动相关性高于人口统计学特征,采用者合并的拉取请求数量比原本多约24%。我们使用合并的拉取请求作为输出的代理指标——承认合并的PR并不等同于其交付的价值——并且这种提升在我们的四个月观察期内持续存在。这些结果表明,命令行编码代理既未被统一采用,也非单纯的新奇效应,组织应将可见的同行使用视为推广策略的核心。

英文摘要:

Organizations rolling out agentic command line tools like Anthropic's Claude Code and GitHub's Copilot CLI need to know who will try them, who will keep using them, and whether the tools produce enough output to justify their cost. At organizational scale, token spend can run into millions of dollars annually, so misreading adoption, retention, or impact can make a rollout expensive without changing engineering velocity. Studying tens of thousands of engineers at Microsoft over its early-2026 rollout, we find that first use spread primarily through social networks, retention was associated more with engineers' coding activity than with demographics, and adopters merged roughly 24% more pull requests than they would have otherwise. We use merged pull requests as our proxy for output -- acknowledging that a merged PR is not the same as the value it delivers -- and the lift persists across our four-month window. These results suggest that CLI coding agents are neither uniformly adopted nor mere novelty effects and that organizations should treat visible peer use as central to rollout strategy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑