DeepMetis：增强深度学习测试集以增加其突变分数
DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score
摘要：
深度学习（DL）组件通常集成到需要执行复杂任务（如图像或自然语言处理）的软件系统中。用于测试此类系统的测试数据的充分性可通过其暴露模拟真实DL故障的人工注入故障（突变）的能力来评估。在本文中，我们描述了一种方法来自动生成新的测试输入，可用于增加现有的测试集，使其检测DL突变的能力增加。我们的工具DeepMetis实现了基于搜索的输入生成策略。为了解释训练和变异过程的非确定性，我们的适应度函数涉及测试中的DL模型的多个实例。实验结果表明，\tool是有效的，在增加给定的测试集，提高了63%的平均能力，以检测突变。一个留一法实验表明，增强的测试集是能够暴露看不见的突变体，模拟尚未检测到的故障的发生。
英文摘要：
Deep Learning (DL) components are routinely integrated into software systems that need to perform complex tasks such as image or natural language processing. The adequacy of the test data used to test such systems can be assessed by their ability to expose artificially injected faults (mutations) that simulate real DL faults. In this paper, we describe an approach to automatically generate new test inputs that can be used to augment the existing test set so that its capability to detect DL mutations increases. Our tool DeepMetis implements a search based input generation strategy. To account for the non-determinism of the training and the mutation processes, our fitness function involves multiple instances of the DL model under test. Experimental results show that \tool is effective at augmenting the given test set, increasing its capability to detect mutants by 63% on average. A leave-one-out experiment shows that the augmented test set is capable of exposing unseen mutants, which simulate the occurrence of yet undetected faults.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
