arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.05738cs.SE
收藏

基于搜索的测试用例生成中超参数调整的实用方法

A Pragmatic Approach for Hyper-Parameter Tuning in Search-based Test Case Generation

Shayan Zamani, Hadi Hemmati

摘要:

基于搜索的测试用例生成是Meta启发式搜索在测试用例生成中的应用,近年来得到了大量的研究。从理论上讲,由于Meta启发式搜索方法的性能高度依赖于其超参数,因此有必要研究该领域的超参数调整。在本文中,我们提出了一个新的度量(“调整增益”),估计如何成本效益调整一个特定的类。然后,我们预测“调优增益”使用源代码类的静态功能。最后,我们根据估计的“调优收益”对类进行优先级调整,并将调优预算仅花费在排名高的类上。为了评估我们的方法,我们详尽地分析了知名的基于搜索的测试生成工具(EvoSuite)的1,200个超参数配置,用于SF 110和SBST 2018工具竞赛等基准测试中的250类19个项目。我们使用了一种称为Meta-GA的调优方法,并比较了有和没有提出的类优先级的调优结果。结果表明,对于低调优预算,优先级类在额外覆盖的分支方面优于替代方案(比传统的全局调优多10倍)。此外,我们报告的影响,我们的方法,如搜索空间大小,调整预算,调整算法,以及调整类的数量,对最终结果的不同功能。

英文摘要:

Search-based test case generation, which is the application of meta-heuristic search for generating test cases, has been studied a lot in the literature, lately. Since, in theory, the performance of meta-heuristic search methods is highly dependent on their hyper-parameters, there is a need to study hyper-parameter tuning in this domain. In this paper, we propose a new metric ("Tuning Gain"), which estimates how cost-effective tuning a particular class is. We then predict "Tuning Gain" using static features of source code classes. Finally, we prioritize classes for tuning, based on the estimated "Tuning Gains" and spend the tuning budget only on the highly-ranked classes. To evaluate our approach, we exhaustively analyze 1,200 hyper-parameter configurations of a well-known search-based test generation tool (EvoSuite) for 250 classes of 19 projects from benchmarks such as SF110 and SBST2018 tool competition. We used a tuning approach called Meta-GA and compared the tuning results with and without the proposed class prioritization. The results show that for a low tuning budget, prioritizing classes outperforms the alternatives in terms of extra covered branches (10 times more than a traditional global tuning). In addition, we report the impact of different features of our approach such as search space size, tuning budgets, tuning algorithms, and the number of classes to tune, on the final results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑