arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.12944cs.SEcs.HC
收藏

AI 对开发速度的影响有多大?一项基于企业的随机对照试验

How much does AI impact development speed? An enterprise-based randomized controlled trial

Elise Paradis, Kate Grey, Quinn Madison, Daye Nam, Andrew Macvean, Vahid Meimand, Nan Zhang, Ben Ferrari-Church, Satish Chandra

AI总结:

本研究通过 96 名谷歌工程师的随机对照试验,发现 AI 辅助显著缩短了复杂企业级任务的开发时间,最佳估计约缩短 21%,并指出每日编码时长越长的开发者受益更明显。

摘要:

AI 辅助对开发者生产力的影响有多大?迄今为止,软件工程文献提供了一系列答案,涉及多种结果:从感知生产力到任务完成速度以及开发者吞吐量。我们与 96 名谷歌全职软件工程师进行的随机对照试验,通过分享三个 AI 功能对开发者完成一项复杂、企业级任务所花费时间影响的估计,为这一文献做出了贡献。我们发现 AI 显著缩短了开发者完成任务所花费的时间。在控制已知影响开发者任务时间的因素后,我们对这一效应规模的最佳估计约为 21%,尽管我们的置信区间较大。我们还发现了一个有趣的效应,即每天在代码相关活动上花费更多小时的开发者在使用 AI 时速度更快。文中讨论了产品及未来研究的考虑。特别是,我们邀请进一步研究,在生态系统层面以及跨多套 AI 增强工具探索 AI 的影响,因为我们不能假设我们实验室研究中获得的效应规模必然更广泛地适用,或者 2024 年夏季使用谷歌内部工具发现的 AI 效应会跨工具并随时间推移而转化。

英文摘要:

How much does AI assistance impact developer productivity? To date, the software engineering literature has provided a range of answers, targeting a diversity of outcomes: from perceived productivity to speed on task and developer throughput. Our randomized controlled trial with 96 full-time Google software engineers contributes to this literature by sharing an estimate of the impact of three AI features on the time developers spent on a complex, enterprise-grade task. We found that AI significantly shortened the time developers spent on task. Our best estimate of the size of this effect, controlling for factors known to influence developer time on task, stands at about 21\%, although our confidence interval is large. We also found an interesting effect whereby developers who spend more hours on code-related activities per day were faster with AI. Product and future research considerations are discussed. In particular, we invite further research that explores the impact of AI at the ecosystem level and across multiple suites of AI-enhanced tools, since we cannot assume that the effect size obtained in our lab study will necessarily apply more broadly, or that the effect of AI found using internal Google tooling in the summer of 2024 will translate across tools and over time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑