arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.05933cs.SE
收藏

基于图的深度学习推理引擎模糊测试

Graph-Based Fuzz Testing for Deep Learning Inference Engine

Weisi Luo, Dong Chai, Xiaoyue Run, Jiang Wang, Chunrong Fang, Zhenyu Chen

摘要:

随着深度学习(DL)系统的广泛应用,学术界和工业界开始关注其质量。测试是质量保证的主要方法之一。然而,现有的测试技术关注DL模型的质量,但缺乏对核心底层推理引擎(即,框架和库)。受模糊测试成功案例的启发,本文设计了一种基于图的模糊测试方法来提高深度学习推理机的测试质量。DL模型的图结构自然遵循这种方法。引入了一种新的基于图论的算子级覆盖准则,并通过探索模型结构、参数和数据输入的组合,实现了六种不同的变异以生成多样化的DL模型。蒙特卡罗树搜索(MCTS)用于在没有训练过程的情况下驱动DL模型生成。实验结果表明,MCTS在提高操作符级覆盖率和检测异常方面优于随机方法。我们的方法已经发现了超过40个不同的异常,在三种类型的不期望的行为:模型转换失败,推理失败,输出比较失败。变异策略是有用的,以产生新的有效的测试输入,高达8.2%以上的运营商级的覆盖率平均和8.6以上的异常捕获。

英文摘要:

With the wide use of Deep Learning (DL) systems, academy and industry begin to pay attention to their quality. Testing is one of the major methods of quality assurance. However, existing testing techniques focus on the quality of DL models but lacks attention to the core underlying inference engines (i.e., frameworks and libraries). Inspired by the success stories of fuzz testing, we design a graph-based fuzz testing method to improve the quality of DL inference engines. This method is naturally followed by the graph structure of DL models. A novel operator-level coverage criterion based on graph theory is introduced and six different mutations are implemented to generate diversified DL models by exploring combinations of model structures, parameters, and data inputs. The Monte Carlo Tree Search (MCTS) is used to drive DL model generation without a training process. The experimental results show that the MCTS outperforms the random method in boosting operator-level coverage and detecting exceptions. Our method has discovered more than 40 different exceptions in three types of undesired behaviors: model conversion failure, inference failure, output comparison failure. The mutation strategies are useful to generate new valid test inputs, by up to 8.2% more operator-level coverage on average and 8.6 more exceptions captured.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑