通用测试用例以实现全面的测试场景覆盖
Generalizing Test Cases for Comprehensive Test Scenario Coverage
AI总结:
本文提出TestGeneralizer框架,通过增强需求理解、生成测试场景模板并生成可执行测试用例,提升测试场景覆盖。在12个开源Java项目上测试,相比ChatTester在突变和LLM评估中分别提升31.66%和23.08%。
摘要:
测试用例对于软件开发和维护至关重要。在实践中,开发者基于对需求的理解和对多样测试场景的推断,从隐含模式中生成多个测试用例,每个验证焦点方法的特定行为。然而,生成全面的测试是耗时且易出错的:许多重要的测试通常在重大延迟后才添加,有时甚至在触发bug之后。现有的自动化测试生成技术主要关注代码覆盖。然而在实际项目中,实用测试很少由代码覆盖单独驱动,因为测试场景不一定与控制流分支一致。相反,测试场景源于需求,这些需求通常未记录且隐含在项目的设计和实现中。然而,开发者编写的测试通常被视为可执行规范;因此,即使一个初始测试反映了开发者的意图,也能揭示底层需求和应验证的多样场景。在本文中,我们提出TestGeneralizer,一个用于通用测试用例以全面覆盖测试场景的框架。TestGeneralizer orchestrate三个阶段:(1) 增强焦点方法和初始测试背后的需求和场景的理解;(2) 生成测试场景模板并将其结晶为各种测试场景实例;(3) 从这些实例中生成并完善可执行测试用例。我们评估TestGeneralizer在12个开源Java项目上,与三个最先进的基线相比,TestGeneralizer在突变和LLM评估的测试场景覆盖上分别提升了31.66%和23.08%。
英文摘要:
Test cases are essential for software development and maintenance. In practice, developers derive multiple test cases from an implicit pattern based on their understanding of requirements and inference of diverse test scenarios, each validating a specific behavior of the focal method. However, producing comprehensive tests is time-consuming and error-prone: many important tests that should have accompanied the initial test are added only after a significant delay, sometimes only after bugs are triggered. Existing automated test generation techniques largely focus on code coverage. Yet in real projects, practical tests are seldom driven by code coverage alone, since test scenarios do not necessarily align with control-flow branches. Instead, test scenarios originate from requirements, which are often undocumented and implicitly embedded in a project's design and implementation. However, developer-written tests are frequently treated as executable specifications; thus, even a single initial test that reflects the developer's intent can reveal the underlying requirement and the diverse scenarios that should be validated. In this work, we propose TestGeneralizer, a framework for generalizing test cases to comprehensively cover test scenarios. TestGeneralizer orchestrates three stages: (1) enhancing the understanding of the requirement and scenario behind the focal method and initial test; (2) generating a test scenario template and crystallizing it into various test scenario instances; and (3) generating and refining executable test cases from these instances. We evaluate TestGeneralizer against three state-of-the-art baselines on 12 open-source Java projects. TestGeneralizer achieves significant improvements: +31.66% and +23.08% over ChatTester, in mutation-based and LLM-assessed scenario coverage, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。