arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.03093cs.SE
收藏

ASTER:利用大语言模型生成自然且多语言的单元测试

ASTER: Natural and Multi-language Unit Test Generation with LLMs

Rangeet Pan, Myeongsoo Kim, Rahul Krishna, Raju Pavuluri, Saurabh Sinha

AI总结:

针对现有单元测试生成工具支持语言少、生成测试可读性差的问题,提出结合静态分析引导LLM的通用测试生成流程,在Java和Python场景下验证其覆盖率优于现有技术,且生成的测试更自然易读。

摘要:

在软件开发中,实现自动化单元测试是一项重要但耗时的工作。为帮助开发者完成这项任务,人们已开发出许多自动化单元测试生成技术。然而,尽管付出了这些努力,可用的工具仅适用于极少数编程语言。此外,研究发现自动生成的测试可读性差,与开发者编写的测试并不相似。\n在本研究中,我们针对大语言模型(LLM)如何助力弥合这一差距开展了严谨的调研。我们提出了一种通用流程,该流程结合静态分析来引导LLM生成可编译且高覆盖率的测试用例。我们阐述了该流程如何应用于不同编程语言(具体为Java和Python),以及需要环境模拟的复杂软件。\n我们开展了一项实证研究,从代码覆盖率和测试自然度两方面评估生成测试的质量——在标准Java应用、企业级Java应用以及一个大型Python基准测试上对其进行评估。研究结果表明,基于LLM的测试生成在静态分析的引导下,在实现的覆盖率上可与最先进的测试生成技术相媲美,甚至更优,同时还能生成自然度高得多、开发者易于理解的测试用例。我们还展示了一项由161名专业开发者参与的用户研究结果,该研究凸显了我们的方法所生成测试的自然度特性。

英文摘要:

Implementing automated unit tests is an important but time-consuming activity in software development. To assist developers in this task, many techniques for automating unit test generation have been developed. However, despite this effort, usable tools exist for very few programming languages. Moreover, studies have found that automatically generated tests suffer poor readability and do not resemble developer-written tests. In this work, we present a rigorous investigation of how large language models (LLMs) can help bridge the gap. We describe a generic pipeline that incorporates static analysis to guide LLMs in generating compilable and high-coverage test cases. We illustrate how the pipeline can be applied to different programming languages, specifically Java and Python, and to complex software requiring environment mocking. We conducted an empirical study to assess the quality of the generated tests in terms of code coverage and test naturalness -- evaluating them on standard as well as enterprise Java applications and a large Python benchmark. Our results demonstrate that LLM-based test generation, when guided by static analysis, can be competitive with, and even outperform, state-of-the-art test-generation techniques in coverage achieved while also producing considerably more natural test cases that developers find easy to understand. We also present the results of a user study, conducted with 161 professional developers, that highlights the naturalness characteristics of the tests generated by our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑