Context. The last several years saw the emergence of AI assistants for code - multi-purpose AI-based helpers in software engineering. As they become omnipresent in all aspects of software development, it becomes critical to understand their usage patterns.
Objective. We aim to better understand how specifically developers are using AI assistants, why they are not using them in certain parts of their development workflow, and what needs to be improved in the future.
Methods. In this work, we carried out a large-scale survey aimed at how AI assistants are used, focusing on specific software development activities and stages. We collected opinions of 481 programmers on five broad activities: (a) implementing new features, (b) writing tests, (c) bug triaging, (d) refactoring, and (e) writing natural-language artifacts, as well as their individual stages.
Results. Our results provide a novel comparison of different stages where AI assistants are used that is both comprehensive and detailed. It highlights specific activities that developers find less enjoyable and want to delegate to an AI assistant, e.g., writing tests and natural-language artifacts. We also determine more granular stages where AI assistants are used, such as generating tests and generating docstrings, as well as less studied parts of the workflow, such as generating test data. Among the reasons for not using assistants, there are general aspects like trust and company policies, as well as more concrete issues like the lack of project-size context, which can be the focus of the future research.
Conclusion. The provided analysis highlights stages of software development that developers want to delegate and that are already popular for using AI assistants, which can be a good focus for features aimed to help developers right now. The main reasons for not using AI assistants can serve as a guideline for future work.
Tokalator: A Context Engineering Toolkit for Artificial Intelligence Coding Assistants
Kariyer.net, R&D Center（Kariyer.net 研发中心）；Stanford University and iLab, Stanford, CA, U.S.A.（斯坦福大学及iLab）；Centre Tecnològic de Telecomunicacions de Catalunya (CTTC/CERCA)（加泰罗尼亚电信技术中心）
The Matthew Effect of AI Programming Assistants: A Hidden Bias in Software Evolution
City University of Hong Kong (Dongguan)（香港城市大学（东莞））；The Hong Kong Polytechnic University（香港理工大学）；The University of Hong Kong（香港大学）；The Hong Kong University of Science and Technology（香港科技大学）
CelloAI Benchmarks: Toward Repeatable Evaluation of AI Assistants
Brookhaven National Laboratory（布鲁克海文国家实验室）；Stony Brook University（石溪大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Argonne National Laboratory（阿贡国家实验室）
Debt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the Wild
Adelaide University（阿德莱德大学）；University of Victoria（维多利亚大学）；Huazhong University of Science and Technology（华中科技大学）；Singapore Management University（新加坡管理大学）
"Always Nice and Confident, Sometimes Wrong": Developer's Experiences Engaging Large Language Models (LLMs) Versus Human-Powered Q&A Platforms for Coding Support
Towards Trustworthy AI Software Development Assistance
Technische Universität Darmstadt（德累斯顿技术大学）；Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心）；National Research Center for Applied Cybersecurity ATHENE（应用网络安全国家研究中心ATHENE）
