RovoDev Code Reviewer: 一个大规模在线评估Atlassian中基于LLM的代码审查自动化工具
RovoDev Code Reviewer: A Large-Scale Online Evaluation of LLM-based Code Review Automation at Atlassian
AI总结:
RovoDev Code Reviewer通过大规模评估展示了基于LLM的代码审查自动化在提升效率和质量方面的效果
摘要:
大型语言模型(LLM)驱动的代码审查自动化有潜力改变代码审查流程。尽管已有许多进展在LLM驱动的代码审查评论生成方法上,但在设计企业级代码审查自动化工具方面仍存在几个实际挑战。特别是,本文旨在回答一个实际问题:如何设计一个具有指导性、上下文感知、质量检查的代码审查评论生成系统,而无需微调?在本文中,我们提出了RovoDev Code Reviewer,这是一个在Atlassian开发生态系统中设计和部署的大型语言模型驱动的代码审查自动化工具,并无缝集成到Atlassian的Bitbucket中。通过一年期间的离线、在线和用户反馈评估,我们得出结论:RovoDev Code Reviewer在生成能够导致代码解决的代码审查评论方面是有效的(即,评论触发了后续提交中的代码更改,占38.70%);并且提供了加快反馈周期(即,将PR周期时间减少30.8%)的潜力,减轻审查员的工作量(即,减少人工撰写的评论数量35.6%),并提高整体软件质量(即,发现具有可操作建议的错误)。
英文摘要:
Large Language Models (LLMs)-powered code review automation has the potential to transform code review workflows. Despite the advances of LLM-powered code review comment generation approaches, several practical challenges remain for designing enterprise-grade code review automation tools. In particular, this paper aims at answering the practical question: how can we design a review-guided, context-aware, quality-checked code review comment generation without fine-tuning? In this paper, we present RovoDev Code Reviewer, an enterprise-grade LLM-based code review automation tool designed and deployed at scale within Atlassian's development ecosystem with seamless integration into Atlassian's Bitbucket. Through the offline, online, user feedback evaluations over a one-year period, we conclude that RovoDev Code Reviewer is effective in generating code review comments that could lead to code resolution for 38.70% (i.e., comments that triggered code changes in the subsequent commits); and offers the promise of accelerating feedback cycles (i.e., decreasing the PR cycle time by 30.8%), alleviating reviewer workload (i.e., reducing the number of human-written comments by 35.6%), and improving overall software quality (i.e., finding errors with actionable suggestions).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。