基于Transformers和焦点上下文的单元测试用例生成
Unit Test Case Generation with Transformers and Focal Context
摘要:
自动化的单元测试用例生成工具促进了测试驱动的开发,并通过建议旨在识别代码中缺陷的测试来支持开发人员。现有的方法通常由测试覆盖标准指导,生成的合成测试用例通常难以为开发人员阅读或理解。在本文中,我们提出了AthenaTest,一种方法,旨在通过学习现实世界的焦点方法和开发人员编写的测试用例生成单元测试的情况下。我们制定单元测试的情况下,生成一个序列到序列的学习任务,采用两步的训练过程,包括去噪预训练的大型无监督Java语料库,和监督微调的下游翻译任务生成单元测试。我们研究了自然语言和源代码预训练的影响,以及焦点方法周围的焦点上下文信息。这两种技术在验证损失方面都有改进,预训练产生了25%的相对改进,焦点上下文提供了额外的11.1%的改进。我们还介绍了Methods 2 Test,这是Java中最大的单元测试用例方法和相应的焦点方法的公开监督并行语料库,其中包括从GitHub的91 K开源存储库中挖掘的780 K测试用例。我们评估了AthenaTest在五个defects 4j项目上的测试,仅用30次尝试就产生了25 K个通过测试的案例,覆盖了43.7%的焦点方法。我们执行测试用例,收集测试覆盖信息,并将它们与EvoSuite和GPT-3生成的测试用例进行比较,发现我们的方法优于GPT-3,并且具有可比的覆盖率。EvoSuite。最后,我们调查了专业开发人员在可读性,可理解性和测试有效性方面的偏好生成的测试,显示出压倒性的偏好AthenaTest。
英文摘要:
Automated unit test case generation tools facilitate test-driven development and support developers by suggesting tests intended to identify flaws in their code. Existing approaches are usually guided by the test coverage criteria, generating synthetic test cases that are often difficult for developers to read or understand. In this paper we propose AthenaTest, an approach that aims to generate unit test cases by learning from real-world focal methods and developer-written testcases. We formulate unit test case generation as a sequence-to-sequence learning task, adopting a two-step training procedure consisting of denoising pretraining on a large unsupervised Java corpus, and supervised finetuning for a downstream translation task of generating unit tests. We investigate the impact of natural language and source code pretraining, as well as the focal context information surrounding the focal method. Both techniques provide improvements in terms of validation loss, with pretraining yielding 25% relative improvement and focal context providing additional 11.1% improvement. We also introduce Methods2Test, the largest publicly available supervised parallel corpus of unit test case methods and corresponding focal methods in Java, which comprises 780K test cases mined from 91K open-source repositories from GitHub. We evaluate AthenaTest on five defects4j projects, generating 25K passing test cases covering 43.7% of the focal methods with only 30 attempts. We execute the test cases, collect test coverage information, and compare them with test cases generated by EvoSuite and GPT-3, finding that our approach outperforms GPT-3 and has comparable coverage w.r.t. EvoSuite. Finally, we survey professional developers on their preference in terms of readability, understandability, and testing effectiveness of the generated tests, showing overwhelmingly preference towards AthenaTest.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。