AI系统测试的分布感知
Distribution Awareness for AI System Testing
摘要:
随着深度学习(DL)在许多安全关键应用中的不断采用,其质量和可靠性开始引起人们的关注。与传统的软件开发过程类似,在早期阶段测试DL软件以发现其缺陷是降低部署后风险的有效方法。尽管最近在设计用于DL软件的新颖测试技术方面取得了进展,但是没有考虑生成的测试数据的分布。因此,很难判断所识别的错误是否确实是对DL应用有意义的错误。因此,我们提出了一种新的面向对象指导的测试技术,其目的是产生新的看不见的测试用例相关的底层DL系统任务。我们的结果表明,该技术是能够过滤高达55.44%的错误的测试用例CIFAR-10和10.05%的有效提高鲁棒性。
英文摘要:
As Deep Learning (DL) is continuously adopted in many safety critical applications, its quality and reliability start to raise concerns. Similar to the traditional software development process, testing the DL software to uncover its defects at an early stage is an effective way to reduce risks after deployment. Although recent progress has been made in designing novel testing techniques for DL software, the distribution of generated test data is not taken into consideration. It is therefore hard to judge whether the identified errors are indeed meaningful errors to the DL application. Therefore, we propose a new OOD-guided testing technique which aims to generate new unseen test cases relevant to the underlying DL system task. Our results show that this technique is able to filter up to 55.44% of error test case on CIFAR-10 and is 10.05% more effective in enhancing robustness.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。