arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.09020cs.SE
收藏

解释“解释”:代码评审中解释行为的实证研究

Explaining Explanation: An Empirical Study on Explanation in Code Reviews

Ratnadira Widyasari, Ting Zhang, Abir Bouraffa, Walid Maalej, David Lo

AI总结:

该研究通过标注793条Gerrit代码评审评论,划分出7类解释并分析其使用差异,验证了ChatGPT指定类型生成解释的准确率达97.8%,为代码评审自动化辅助提供基础。

摘要:

代码评审是软件质量保障的核心环节。理想情况下,评审者应当对其反馈作出解释,以便代码变更的作者理解反馈并采取相应行动。不同开发者在不同场景下可能需要不同的解释。因此,要为这一流程提供辅助,首先需要明确评审者通常提供的解释类型。本文的研究目标是探究代码评审中使用的解释类型,并探索大语言模型(LLM),具体为ChatGPT,生成这些特定类型解释的潜力。我们从Gerrit中提取了793条代码评审评论,并根据其是否包含建议、解释或二者兼有进行人工标注。分析显示,42%的评论仅包含建议而无解释。我们将解释划分为7种不同类型,包括规则或原则、相似示例、未来影响等。在统计各类解释的流行度时,我们发现新手评审者和经验丰富的评审者对部分解释类型的使用情况存在差异。人工评估结果表明,当指定解释类型时,ChatGPT在90个案例中有88个能正确生成对应解释。这项基础工作凸显了代码评审未来自动化的潜力,可辅助开发者按需分享和获取不同类型的解释,从而减少来回沟通成本。

英文摘要:

Code reviews are central for software quality assurance. Ideally, reviewers should explain their feedback to enable authors of code changes to understand the feedback and act accordingly. Different developers might need different explanations in different contexts. Therefore, assisting this process first requires understanding the types of explanations reviewers usually provide. The goal of this paper is to study the types of explanations used in code reviews and explore the potential of Large Language Models (LLMs), specifically ChatGPT, in generating these specific types. We extracted 793 code review comments from Gerrit and manually labeled them based on whether they contained a suggestion, an explanation, or both. Our analysis shows that 42% of comments only include suggestions without explanations. We categorized the explanations into seven distinct types including rule or principle, similar examples, and future implications. When measuring their prevalence, we observed that some explanations are used differently by novice and experienced reviewers. Our manual evaluation shows that, when the explanation type is specified, ChatGPT can correctly generate the explanation in 88 out of 90 cases. This foundational work highlights the potential for future automation in code reviews, which can assist developers in sharing and obtaining different types of explanations as needed, thereby reducing back-and-forth communication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑