arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.20004cs.SE
收藏

基于需求覆盖的测试用例最小化

Requirements Coverage-Guided Minimization for Natural Language Test Cases

University of Ottawa(渥太华大学);Research Ireland Lero Centre, University of Limerick(利默里克大学 Research Ireland Lero 中心);Wind River Systems(风河系统公司)

Rongqi Pan, Feifei Niu, Lionel C. Briand, Hanyang Hu

AI总结:

本文提出RTM方法,通过需求覆盖引导的测试套件最小化,有效减少冗余测试用例并保持高故障检测率。

摘要:

随着软件系统的不断发展,测试套件往往在规模上不断扩大,通常包含冗余的测试用例。这种冗余会增加测试的工作量、时间和成本。测试套件最小化(TSM)旨在消除这种冗余,同时保持关键属性,如需求覆盖和故障检测能力。在本文中,我们提出RTM(需求覆盖引导的测试套件最小化),一种新的TSM方法,专为基于需求的测试(验证)设计,能够有效减少测试套件的冗余,同时确保完整的需求数覆盖和在固定最小化预算下的高故障检测率(FDR)。基于在关键系统中功能安全重要的常见实践,我们假设测试用例是在自然语言中指定并追溯到需求后再进行实现的。RTM使用三种不同的预处理方法对测试用例进行预处理,然后使用七种文本嵌入技术将它们转换为向量表示。通过三种距离函数计算向量之间的相似性值。然后使用一个遗传算法,其种群由保持覆盖的初始化策略初始化,用于识别一个包含多样化测试用例的优化子集,以匹配集预算。我们评估了RTM在包含736个系统测试用例和54个需求的工业汽车系统数据集上的表现。实验结果表明,RTM在不同最小化预算下始终优于基线技术,在保持完整需求数覆盖的同时,FDR更高。此外,我们研究了测试套件冗余水平对TSM有效性的影响,提供了在实际约束下优化基于需求的测试套件的新见解。

英文摘要:

As software systems evolve, test suites tend to grow in size and often contain redundant test cases. Such redundancy increases testing effort, time, and cost. Test suite minimization (TSM) aims to eliminate such redundancy while preserving key properties such as requirement coverage and fault detection capability. In this paper, we propose RTM (Requirement coverage-guided Test suite Minimization), a novel TSM approach designed for requirement-based testing (validation), which can effectively reduce test suite redundancy while ensuring full requirement coverage and a high fault detection rate (FDR) under a fixed minimization budget. Based on common practice in critical systems where functional safety is important, we assume test cases are specified in natural language and traced to requirements before being implemented. RTM preprocesses test cases using three different preprocessing methods, and then converts them into vector representations using seven text embedding techniques. Similarity values between vectors are computed utilizing three distance functions. A Genetic Algorithm, whose population is initialized by coverage-preserving initialization strategies, is then employed to identify an optimized subset containing diverse test cases matching the set budget. We evaluate RTM on an industrial automotive system dataset comprising $736$ system test cases and $54$ requirements. Experimental results show that RTM consistently outperforms baseline techniques in terms of FDR across different minimization budgets while maintaining full requirement coverage. Furthermore, we investigate the impact of test suite redundancy levels on the effectiveness of TSM, providing new insights into optimizing requirement-based test suites under practical constraints.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑