arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.18531cs.SE
收藏

实践中的自动化代码审查

Automated Code Review In Practice

Umut Cihan, Vahid Haratian, Arda İçöz, Mert Kaan Gül, Ömercan Devran, Emircan Furkan Bayendur, Baykal Mehmet Uçar, Eray Tüzün

AI总结:

本文通过分析工业环境中基于LLM的自动化代码审查工具的使用数据与开发者反馈,发现该工具虽能提升缺陷检测和代码质量意识,但也导致拉取请求关闭时间延长并引入错误审查等问题。

摘要:

代码审查是一种广泛采用的实践,用于提高软件质量并传递知识。由于需要人工投入且可能存在延迟,代码审查通常被视为耗时的工作。一些AI辅助工具,如Qodo、GitHub Copilot和Coderabbit,利用大型语言模型(LLMs)提供自动化审查。此类工具在工业界的效果尚待检验。\n本研究考察了基于LLM的自动化代码审查工具在工业环境中的影响。该研究在一个采用了AI辅助审查工具(基于开源的Qodo PR Agent)的软件开发环境中进行。来自十个项目的大约238名从业者可以使用该工具。我们重点关注了三个项目,共涉及4,335个拉取请求,其中1,568个经过了自动化审查。数据收集包括三个来源:(1)对拉取请求数据的定量分析,包括表明开发者是否对自动化评论采取行动的评论标签;(2)向开发者发送的关于他们对单个拉取请求审查体验的调查问卷;(3)对22名从业者进行的更广泛的调查,以获取他们对自动化审查的总体看法。\n73.8%的自动化评论得到了解决。然而,拉取请求的平均关闭时长从5小时52分钟增加到8小时20分钟,且不同项目之间的趋势存在差异。大多数从业者报告称,自动化审查使代码质量略有改善。\n基于LLM的工具在软件开发中被证明是有用的,它增强了缺陷检测能力,提高了对代码质量的意识,并促进了最佳实践。然而,它也导致了更长的拉取请求关闭时间,并带来了诸如错误审查、不必要的更正和不相关评论等缺点。

英文摘要:

Code review is a widespread practice to improve software quality and transfer knowledge. It is often seen as time-consuming due to the need for manual effort and potential delays. Several AI-assisted tools, such as Qodo, GitHub Copilot, and Coderabbit, provide automated reviews using large language models (LLMs). The effects of such tools in the industry are yet to be examined. This study examines the impact of LLM-based automated code review tools in an industrial setting. The study was conducted within a software development environment that adopted an AI-assisted review tool (based on open-source Qodo PR Agent). Around 238 practitioners across ten projects had access to the tool. We focused on three projects with 4,335 pull requests, 1,568 of which underwent automated reviews. Data collection comprised three sources: (1) a quantitative analysis of pull request data, including comment labels indicating whether developers acted on the automated comments, (2) surveys sent to developers regarding their experience with reviews on individual pull requests, and (3) a broader survey of 22 practitioners capturing their general opinions on automated reviews. 73.8% of automated comments were resolved. However, the average pull request closure duration increased from five hours 52 minutes to eight hours 20 minutes, with varying trends across projects. Most practitioners reported a minor improvement in code quality due to automated reviews. The LLM-based tool proved useful in software development, enhancing bug detection, increasing awareness of code quality, and promoting best practices. However, it also led to longer pull request closure times and introduced drawbacks like faulty reviews, unnecessary corrections, and irrelevant comments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑