AI 中文总结
研究针对深度神经网络变异测试成本高的问题,提出Mure框架,通过记忆化仅执行突变体变异后缀并重用原始模型公共前缀,实现可证明无损加速,实验证明其能有效降低计算成本,在高变异率下也有显著加速效果。
AI 中文摘要
变异分析作为评估测试数据集充分性的一种有前景但成本高昂的方法,在深度神经网络(DNN)中重新出现。现有技术通过有损近似加速DNN变异测试,以效率换取变异分数准确性。本文介绍了Mure,首个通过记忆化加速DNN变异测试的可证明无损框架。Mure基于DNN突变体与原始模型共享大量冗余计算的理念,在变异测试时仅执行每个突变体的变异后缀并重用原始模型的公共前缀,且仅计算一次。给出记忆化变异测试的形式化说明,证明Mure合理,即产生与详尽的普通变异测试等效的结果,并确定保证加速的基本条件。实现了Mure并在15个不同架构、复杂度和规模(参数从数千到数百万)的DNN模型上进行评估。实证证据表明,Mure平均将变异测试的计算成本降低44.54%。还观察到,虽然现有技术往往能实现更高的加速(平均高达88.97%),但会以变异分数的一些误差为代价。进一步分析变异生成选择率对Mure有效性的影响,发现随着变异神经元百分比增加,记忆化机会可预测地减少。即使高达5%的神经元发生变异,Mure仍能提供超过20%的加速。
英文摘要
Mutation analysis has recently reemerged in the context of deep neural networks (DNNs) as a promising, but notoriously costly, approach for assessing test dataset adequacy. Existing techniques speed up DNN mutation testing through lossy approximations that trade efficiency for mutation score accuracy. This paper introduces Mure, the first provably lossless framework for accelerating DNN mutation testing via memoization. Mure is based on the idea that DNN mutants and the original model share substantial redundant computation, so during mutation testing, it executes only the mutated suffixes of each mutant and reuses the common prefix from the original model, which is computed only once. We give a formal account of memoized mutation testing, and prove that Mure is sound, i.e., it produces results equivalent to exhaustive vanilla mutation testing, and identify basic conditions under which speed-up is guaranteed. We have implemented Mure and evaluated it on 15 DNN models of various architectures, complexities, and sizes ranging from a few thousands to millions of parameters. This provides empirical evidence that Mure reduces the computational cost of mutation testing by 44.54%, on average. We also observed that while state-of-the-art techniques tend to yield higher acceleration (up to 88.97%, on average), they come at the cost of some error in mutation score. We further analyze the effect of mutation generation selection ratio on the effectiveness of Mure and observed predictable reductions in memoization opportunities with increasing the percentage of mutated neurons. We observed that Mure offers more than 20% speed-up even when as high as 5% of the neurons are mutated.
CommentsProceedings of 35th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2026)
DOI:10.1145/3832252