Code review is an effective software quality assurance activity; however, it is labor-intensive and time-consuming. Thus, a number of generation-based automatic code review (ACR) approaches have been proposed recently, which leverage deep learning techniques to automate various activities in the code review process (e.g., code revision generation and review comment generation).
We find the previous works carry three main limitations. First, the ACR approaches have been shown to be beneficial in each work, but those methods are not comprehensively compared with each other to show their superiority over their peer ACR approaches. Second, general-purpose pre-trained models such as CodeT5 are proven to be effective in a wide range of Software Engineering (SE) tasks. However, no prior work has investigated the effectiveness of these models in ACR tasks yet. Third, prior works heavily rely on the Exact Match (EM) metric which only focuses on the perfect predictions and ignores the positive progress made by incomplete answers. To fill such a research gap, we conduct a comprehensive study by comparing the effectiveness of recent ACR tools as well as the general-purpose pre-trained models. The results show that a general-purpose pre-trained model CodeT5 can outperform other models in most cases. Specifically, CodeT5 outperforms the prior state-of-the-art by 13.4\%--38.9\% in two code revision generation tasks. In addition, we introduce a new metric namely Edit Progress (EP) to quantify the partial progress made by ACR tools. The results show that the rankings of models for each task could be changed according to whether EM or EP is being utilized. Lastly, we derive several insightful lessons from the experimental results and reveal future research directions for generation-based code review automation.
Can Commercial Testing Automation Tools Work for IoT? A Case Study of Selenium and Node-Red
背景:由于大规模、数据量和异构性,测试物联网软件具有挑战性。测试自动化是该领域中非常需要的特性。目的:本研究的第一个目标是探索物联网测试自动化的需求和挑战。第二个目标是将商业软件中使用的测试自动化工具集成到物联网环境中。方法:进行了系统的文献综述,以引出物联网测试自动化的要求。遵循设计科学方法,使用商业测试自动化工具Selenium为Node-Red平台编写的物联网应用程序构建测试自动化工具。由此产生的框架使用Selenium Web Driver,用于物联网应用程序的基于浏览器的测试自动化。结果如下:该框架已在多个浏览器上进行了功能测试,并对可维护性、浏览器性能和全面性进行了初步评估。结论:使用商业工具进行物联网自动化测试是可行的。然而,需要全面解决大数据量和数据的并行传输和处理等主要挑战,以实现完全集成。
LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning
Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Software, Tsinghua University(清华大学软件学院);Sinosoft Company Limited(中科软科技股份有限公司)
Closed-Loop Autonomous Software Development via Jira-Integrated Backlog Orchestration: A Case Study in Deterministic Control and Safety-Constrained Automation
AutoResearch: An Execution-Grounded Multi-Agent Framework for Reliable Research Workflow Automation
International Research Center for Complexity Sciences(国际复杂性科学研究中心);Hangzhou International Innovation Institute, Beihang University(杭州国际创新研究院,北京航空航天大学);Department of Computer Science, College of Science, Mathematics and Technology(科学、数学与技术学院计算机科学系);Wenzhou-Kean University(温州-凯恩大学);Computer Systems Engineering Department, Sukkur IBA University(苏库尔IBA大学计算机系统工程系);Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China(长江三角洲地区研究所(湖州),中国电子科学科技大学)