arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.00436cs.SEcs.HC

通过聊天编程:11579个真实世界AI辅助IDE会话的大规模行为分析

Programming by Chat: A Large-Scale Behavioral Analysis of 11,579 Real-World AI-Assisted IDE Sessions

University of Notre Dame(圣母大学);Vanderbilt University(范德堡大学)

Ningzhi Tang, Chaoran Chen, Zihan Fang, Gelei Xu, Maria Dhakal, Yiyu Shi, Collin McMillan, Yu Huang, Toby Jia-Jun Li

AI总结:

研究通过分析11579个真实IDE会话中的74998条开发者消息,揭示了编程工作组织的三大转变:对话编程作为逐步规范、认知工作分配给AI以及主动管理协作。

摘要:

集成AI代码助手的IDE,能够在开发者的代码库中进行对话式操作,具有项目上下文和多文件编辑能力,正在迅速改变软件开发。然而,对这一转变的经验研究仍有限:现有研究大多依赖小规模受控环境或分析通用聊天机器人而非代码库感知的IDE工作流。我们提出,到目前为止,这是首次对真实世界对话式编程在IDE原生环境中的大规模研究,分析了11579个会话中74998条开发者消息,覆盖1300个仓库和899名开发者,使用Cursor和GitHub Copilot。这些聊天被提交到公共仓库作为常规开发的一部分,捕捉了真实世界的行为。我们的发现揭示了编程工作组织的三大转变:对话编程作为逐步规范,开发者迭代细化输出而非提前规定完整任务;开发者将认知工作分配给AI,委托诊断、理解和验证,而非直接参与代码和输出;开发者积极管理协作,将计划外化为持久化 artifact,并通过上下文注入和行为约束协商AI自主权。这些结果为AI辅助开发提供了基础经验洞察,并为未来编程环境的设计提供了启示。

英文摘要:

IDE-integrated AI coding assistants, which operate conversationally within developers' working codebases with access to project context and multi-file editing, are rapidly reshaping software development. However, empirical investigation of this shift remains limited: existing studies largely rely on small-scale, controlled settings or analyze general-purpose chatbots rather than codebase-aware IDE workflows. We present, to the best of our knowledge, the first large-scale study of real-world conversational programming in IDE-native settings, analyzing 74,998 developer messages from 11,579 chat sessions across 1,300 repositories and 899 developers using Cursor and GitHub Copilot. These chats were committed to public repositories as part of routine development, capturing in-the-wild behavior. Our findings reveal three shifts in how programming work is organized: conversational programming operates as progressive specification, with developers iteratively refining outputs rather than specifying complete tasks upfront; developers redistribute cognitive work to AI, delegating diagnosis, comprehension, and validation rather than engaging with code and outputs directly; and developers actively manage the collaboration, externalizing plans into persistent artifacts, and negotiating AI autonomy through context injection and behavioral constraints. These results provide foundational empirical insights into AI-assisted development and offer implications for the design of future programming environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑