arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1803.04792cs.LGcs.CVcs.SE
收藏

测试深度神经网络

Testing Deep Neural Networks

University of Oxford(牛津大学);University of Liverpool(利物浦大学);Defence Science and Technology Laboratory (Dstl)(国防科学技术实验室(Dstl))

Youcheng Sun, Xiaowei Huang, Daniel Kroening, James Sharp, Matthew Hill, Rob Ashmore

摘要:

深度神经网络(DNN)具有广泛的应用,使用它们的软件必须经过彻底的测试,特别是在安全关键领域。然而,传统的软件测试覆盖度量不能直接应用于DNN。在本文中,受MC/DC覆盖标准的启发,我们提出了一系列四个新的测试标准,这些标准是针对DNN的结构特征及其语义而量身定制的。我们通过证明通过我们提出的覆盖标准指导生成的测试输入能够捕获DNN中的不期望行为来验证标准。测试用例的生成采用符号化方法和基于梯度的启发式搜索。通过将它们与现有方法进行比较,我们表明我们的标准在发现错误(使用对抗性示例进行代理)的能力和测试用例生成的计算成本之间实现了平衡。我们的实验是在使用流行的开源数据集(包括MNIST、CIFAR-10和ImageNet)获得的最先进的DNN上进行的。

英文摘要:

Deep neural networks (DNNs) have a wide range of applications, and software employing them must be thoroughly tested, especially in safety-critical domains. However, traditional software test coverage metrics cannot be applied directly to DNNs. In this paper, inspired by the MC/DC coverage criterion, we propose a family of four novel test criteria that are tailored to structural features of DNNs and their semantics. We validate the criteria by demonstrating that the generated test inputs guided via our proposed coverage criteria are able to capture undesired behaviours in a DNN. Test cases are generated using a symbolic approach and a gradient-based heuristic search. By comparing them with existing methods, we show that our criteria achieve a balance between their ability to find bugs (proxied using adversarial examples) and the computational cost of test case generation. Our experiments are conducted on state-of-the-art DNNs obtained using popular open source datasets, including MNIST, CIFAR-10 and ImageNet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑