arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.09755cs.SE
收藏

测试仪器对测试不稳定性的影响

On the Effect of Instrumentation on Test Flakiness

Shawn Rasheed, Jens Dietrich, Amjed Tahir

摘要:

测试不稳定性是一个影响测试和依赖它的过程的问题。有几个因素会导致或影响测试结果的不稳定性。测试执行顺序、随机性和并发性是一些更常见和研究得更好的原因。一些研究提到代码插装是导致或影响测试脆弱性的一个因素。然而,这一问题的证据很少。在这项研究中,我们试图系统地收集证据的仪器测试片状的影响。我们对Java程序的常见插装类型进行了实验-即应用程序性能监控、覆盖率和分析插装。然后,我们研究了一组九个程序的仪器的影响,这些程序是从用于研究测试片状性的现有数据集获得的,这些数据集由用Java编写的流行GitHub项目组成。我们观察到的情况下,在现实世界中的仪器导致flakiness的程序。然而,这种效果是罕见的。我们还讨论了一个相关的问题-如何仪器可能会干扰片状检测和预防。

英文摘要:

Test flakiness is a problem that affects testing and processes that rely on it. Several factors cause or influence the flakiness of test outcomes. Test execution order, randomness and concurrency are some of the more common and well-studied causes. Some studies mention code instrumentation as a factor that causes or affects test flakiness. However, evidence for this issue is scarce. In this study, we attempt to systematically collect evidence for the effects of instrumentation on test flakiness. We experiment with common types of instrumentation for Java programs - namely, application performance monitoring, coverage and profiling instrumentation. We then study the effects of instrumentation on a set of nine programs obtained from an existing dataset used to study test flakiness, consisting of popular GitHub projects written in Java. We observe cases where real-world instrumentation causes flakiness in a program. However, this effect is rare. We also discuss a related issue - how instrumentation may interfere with flakiness detection and prevention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑