arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.12733cs.SE
收藏

测试集约简与回归测试选择策略的交互作用

On the Interaction between Test-Suite Reduction and Regression-Test Selection Strategies

Sebastian Ruland, Malte Lochau

摘要:

单元测试是软件开发中最成熟的质量保证技术之一。单元测试的一个主要优点是效率(即,测试工作)和有效性(即,故障检测概率)。为此,已经提出了各种策略来利用这种权衡。特别是,测试套件减少(TSR)减少了测试用例的数量(可能是冗余的),同时测试一个单一的程序版本。回归测试选择(RTS)为测试连续的程序修订选择测试用例。然而,TSR和RTS在组合使用时可能会影响-甚至阻碍-彼此的性能。例如,在TSR期间针对特定程序版本丢弃的测试用例可能再次与RTS相关。然而,找到一个两种策略的组合,导致一个合理的权衡整个版本历史的程序是一个悬而未决的问题。本文的目的是更好地了解TSR和RTS之间的相互作用,效率和效果。为此,我们提出了一个可配置的框架,称为RegreTS自动单元测试的C程序。该框架包括TSR和RTS的不同策略及其可能的组合。我们将此框架应用于一系列主题系统,提供了几个重要的见解。首先,TSR几乎总是对RTS的有效性产生负面影响,但对效率产生积极影响。其次,向测试人员揭示连续程序版本之间程序修改的影响的测试用例比简单覆盖修改的代码部分的测试用例有效得多,但会导致更多的测试工作。

英文摘要:

Unit testing is one of the most established quality-assurance techniques for software development. One major advantage of unit testing is the adjustable trade-off between efficiency (i.e., testing effort) and effectiveness (i.e., fault-detection probability). To this end, various strategies have been proposed to exploit this trade-off. In particular, test-suite reduction (TSR) reduces the number of (presumably redundant) test cases while testing a single program version. Regression-test selection (RTS) selects test cases for testing consecutive program revisions. However, both TSR and RTS may influence -- or even obstruct -- each others' performance when used in combination. For instance, test cases discarded during TSR for a particular program version may become relevant again for RTS. However, finding a combination of both strategies leading to a reasonable trade-off throughout the version history of a program is an open question. The goal of this paper is to gain a better understanding of the interactions between TSR and RTS with respect to efficiency and effectiveness. To this end, we present a configurable framework called RegreTS for automated unit-testing of C programs. The framework comprises different strategies for TSR and RTS and possible combinations thereof. We apply this framework to a collection of subject systems, delivering several crucial insights. First, TSR has almost always a negative impact on the effectiveness of RTS, yet a positive impact on efficiency. Second, test cases revealing to testers the effect of program modifications between consecutive program versions are far more effective than test cases simply covering modified code parts, yet causing much more testing effort.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑