Unit testing is crucial for detecting bugs in individual program units but consumes time and effort. Recently, large language models (LLMs) have demonstrated remarkable capabilities in generating unit test cases. However, several problems limit their ability to generate high-quality unit test cases: (1) compilation and runtime errors caused by the hallucination of LLMs; (2) lack of testing and coverage feedback information restricting the increase of code coverage;(3) the repetitive suppression problem causing invalid LLM-based repair and generation attempts. To address these limitations, we propose TestART, a novel unit test generation method. TestART improves LLM-based unit testing via co-evolution of automated generation and repair iteration, representing a significant advancement in automated unit test generation. TestART leverages the template-based repair strategy to effectively fix bugs in LLM-generated test cases for the first time. Meanwhile, TestART extracts coverage information from successful test cases and uses it as coverage-guided testing feedback. It also incorporates positive prompt injection to prevent repetition suppression, thereby enhancing the sufficiency of the final test case. This synergy between generation and repair elevates the correctness and sufficiency of the produced test cases significantly beyond previous methods. In comparative experiments, TestART demonstrates an 18% improvement in pass rate and a 20% enhancement in coverage across three types of datasets compared to baseline models. Additionally, it achieves better coverage rates than EvoSuite with only half the number of test cases. These results demonstrate TestART's superior ability to produce high-quality unit test cases by harnessing the power of LLMs while overcoming their inherent flaws.
Simula Research Laboratory(Simula研究实验室);Oslo Metropolitan University(奥斯陆都市大学);Lero & Trinity College Dublin, School of Computer Science and Statistics(Lero与都柏林三一学院计算机科学与统计学院);National Institute of Informatics(国立情报学研究所)
RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair
State Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);Human Language Technology Research Institute, University of Texas at Dallas(人机语言技术研究院,德克萨斯大学达拉斯分校)
TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code
School of Computer Science and Technology, Chongqing University of Posts and Telecommunications(重庆邮电大学计算机科学与技术学院);Nanyang Technological University(南洋理工大学);Beihang University(北京航空航天大学);Southwest University(西南大学)
Scaling Test-Driven Code Generation from Functions to Classes: An Empirical Study
Chengdu Institute of Computer Applications, Chinese Academy of Sciences(中国科学院成都计算机应用研究所);University of Chinese Academy of Sciences(中国科学院大学);Institute of Multidisciplinary Research for Advanced Materials (IMRAM), Tohoku University(东北大学先进材料跨学科研究所);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology(深圳高等研究院人工智能研究所)
Testuary: Agentic Test Generation with Zero-Shot Parallelism
Johannes Gutenberg University Mainz(约翰内斯·古腾堡美因茨大学);paluno – The Ruhr Institute for Software Technology, University of Duisburg-Essen(杜伊斯堡-埃森大学鲁尔软件技术研究所)
LLM-based Low-Level Integration Test Generation for Java
Lero, the Research Ireland Centre for Software, University of Limerick(利默里克大学 Lero 爱尔兰软件研究中心);University of Ottawa(渥太华大学);the State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);Software Engineering Application Technology Lab Huawei(华为软件工程应用技术实验室)