对人工生成的等价变异体的实证评估
An Empirical Evaluation of Manually Created Equivalent Mutants
AI总结:
本研究针对变异测试中等价变异体的人工生成与识别问题,在Code Defenders游戏场景下开展实证研究,发现人工生成变异体的等价率不足10%,且多数开发者难以准确识别等价变异体,凸显了优化检测机制和开展相关培训的必要性。
摘要:
变异测试旨在评估测试套件在检测源代码中人工植入缺陷方面的有效性,并指导测试套件的改进。尽管变异测试工具在实践中被越来越多地采用,但等价变异体(即仅在语法上不同而语义上无差异的变异体)会阻碍这一过程。\n此前的研究已经调查了变异测试工具产生等价变异体的频率,以及现有检测这些等价变异体的方法的有效性,但目前尚不清楚人类生成等价变异体的程度,以及人类识别这些变异体的表现如何。因此,我们在Code Defenders(一款变异测试游戏,玩家在其中竞争性地生成变异体和测试用例)的场景下研究这些问题。\n通过人工检查和自动化识别方法,我们确定人工生成的变异体中只有不到10%是等价的。令人惊讶的是,我们的发现表明,很大一部分开发者难以准确识别等价变异体,这凸显了改进检测机制以及对开发者进行变异测试培训的必要性。
英文摘要:
Mutation testing consists of evaluating how effective test suites are at detecting artificially seeded defects in the source code, and guiding the improvement of the test suites. Although mutation testing tools are increasingly adopted in practice, equivalent mutants, i.e., mutants that differ only in syntax but not semantics, hamper this process. While prior research investigated how frequently equivalent mutants are produced by mutation testing tools and how effective existing methods of detecting these equivalent mutants are, it remains unclear to what degree humans also create equivalent mutants, and how well they perform at identifying these. We therefore study these questions in the context of Code Defenders, a mutation testing game, in which players competitively produce mutants and tests. Using manual inspection as well as automated identification methods we establish that less than 10 % of manually created mutants are equivalent. Surprisingly, our findings indicate that a significant portion of developers struggle to accurately identify equivalent mutants, emphasizing the need for improved detection mechanisms and developer training in mutation testing.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。