利用大型语言模型进行精选代码审查
Harnessing Large Language Models for Curated Code Reviews
AI总结：
本文提出一种基于大型语言模型的数据集精选流水线，用于提升代码审查数据集质量，并验证其能改善评论生成与代码精化等下游任务表现。
摘要：
在代码审查中，生成结构化且相关的评论对于识别代码问题、促进准确的代码修改以及确保高效的代码审查流程至关重要。精心编写的评论不仅能简化代码审查本身，而且对后续任务（如代码精化，即修改代码以满足输入的审查评论）也必不可少。尽管已有多种基于人工智能的方法旨在实现评论生成自动化，但其有效性仍受限于训练数据的质量。现有的代码审查数据集往往噪声较多且未经精炼，限制了人工智能模型的学习潜力，并阻碍了自动化进程。为应对这些挑战，我们提出了一种精选流水线，旨在提升最大的公开可用代码审查数据集的质量。我们首先建立了一个评估框架，纳入具体标准和类别，以实证研究该数据集的初始质量。随后，我们采用由大型语言模型（LLM）驱动的方法，应用我们的精选流水线对数据集进行精炼。基于同一评估框架，对新精选数据集进行的比较分析表明，评论的清晰度和简洁性得到了显著改善。此外，我们评估了精选数据集对自动化下游任务（特别是评论生成和代码精化）的影响。我们的研究结果表明，精选数据集能够提升模型性能，使其生成更准确的评论。精选后的评论也更有用，因为它们能带来更准确的代码精化。
英文摘要：
In code review, generating structured and relevant comments is crucial for identifying code issues and facilitating accurate code changes that ensure an efficient code review process. Well-crafted comments not only streamline the code review itself but are also essential for subsequent tasks like code refinement, where the code is modified to satisfy the input review comment. Although various AI-based approaches aimed to automate comment generation, their effectiveness remains limited by the quality of the training data. Existing code review datasets are often noisy and unrefined, posing limitations to the learning potential of AI models and hindering the automation process. To address these challenges, we propose a curation pipeline designed to enhance the quality of the largest publicly available code review dataset. We begin by establishing an evaluation framework, incorporating specific criteria and categories to empirically study the initial quality of the dataset. Using a large language model (LLM)-driven approach, we then apply our curation pipeline to refine the dataset. A comparative analysis of the newly curated dataset, based on the same evaluation framework, demonstrates substantial improvements in the clarity and conciseness of the comments. Additionally, we assess the impact of the curated dataset on automating downstream tasks, specifically comment generation and code refinement. Our findings show that the curated dataset leads to enhanced model performance in generating more accurate comments. Curated comments are also more useful as they lead to more accurate code refinement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
