arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1901.01541cs.SE
收藏

多独立目标测试集生成算法(MIO)

Many Independent Objective (MIO) Algorithm for Test Suite Generation

Andrea Arcuri

摘要:

自动生成测试套件本质上是一个多目标问题,因为任何测试目标(例如,要执行的语句或要杀死的突变体)本身就是一个目标。测试套件生成的特点是完全不同于其他更经常的优化问题。例如,给定一个现有的测试套件,可以添加更多的测试来覆盖剩余的目标。人们希望最小数量的小测试能够覆盖尽可能多的目标,但是与首先覆盖这些目标相比,这是一个次要目标。此外,软件测试中的目标数量可能很快变得难以管理,数量级为(数万/数十万),特别是对于工业规模系统的系统测试。传统的多目标优化算法已经可以开始与四个或五个目标进行优化。为了克服这些问题,已经提出了不同的技术,例如整体测试套件(WTS)方法和多目标排序算法(MOSA)。然而,这些技术可能无法很好地扩展到非常大量的目标和有限的搜索预算(系统测试中的典型情况)。在本文中,我们提出了一种新的算法,称为多独立目标(MIO)算法。该算法是根据测试用例集生成的特点设计和裁剪的。在一组人工软件和实际软件上的实证研究表明,MIO算法可以实现更高的覆盖率相比,WTS和MOSA,因为它可以更好地利用测试套件生成的特点。

英文摘要:

Automatically generating test suites is intrinsically a multi-objective problem, as any of the testing targets (e.g, statements to execute or mutants to kill) is an objective on its own. Test suite generation has peculiarities that are quite different from other more regular optimisation problems. For example, given an existing test suite, one can add more tests to cover the remaining objectives. One would like the smallest number of small tests to cover as many objectives as possible, but that is a secondary goal compared to covering those targets in the first place. Furthermore, the amount of objectives in software testing can quickly become unmanageable, in the order of (tens/hundreds of) thousands, especially for system testing of industrial size systems. Traditional multi-objective optimisation algorithms can already start to struggle with just four or five objectives to optimize. To overcome these issues, different techniques have been proposed, like for example the Whole Test Suite (WTS) approach and the Many-Objective Sorting Algorithm (MOSA). However, those techniques might not scale well to very large numbers of objectives and limited search budgets (a typical case in system testing). In this paper, we propose a novel algorithm, called Many Independent Objective (MIO) algorithm. This algorithm is designed and tailored based on the specific properties of test suite generation. An empirical study, on a set of artificial and actual software, shows that the MIO algorithm can achieve higher coverage compared to WTS and MOSA, as it can better exploit the peculiarities of test suite generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑