arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.04351cs.SE
收藏

模糊测试深度学习库中的自动微分

Fuzzing Automatic Differentiation in Deep-Learning Libraries

Chenyuan Yang, Yinlin Deng, Jiayi Yao, Yuxing Tu, Hanchi Li, Lingming Zhang

AI总结:

针对深度学习库中自动微分组件测试不足的问题,提出首个API级模糊测试器$\ abla$Fuzz,利用差分测试一阶及高阶梯度并过滤数值不稳定性误报,在四大主流库中检测出173个缺陷,显著优于现有方法。

摘要:

近年来,深度学习(DL)引起了广泛关注并被广泛部署。因此,越来越多的研究致力于测试DL库和框架。然而,现有工作在很大程度上忽略了任何DL系统的一个关键组件,即自动微分(AD),它是近期DL发展的基础。为此,我们提出了$\ abla$Fuzz,这是首个专门针对DL库中关键AD组件的通用且实用的方法。我们的核心见解是,每个DL库API都可以抽象为一个处理张量/向量的函数,可以在各种执行场景下(使用不同实现计算输出/梯度)对其进行差分测试。我们将$\ abla$Fuzz实现为一个针对DL库中AD的完全自动化的API级模糊测试器,它利用不同执行场景下的差分测试来测试一阶和高阶梯度,并包含自动化过滤策略以消除由数值不稳定性引起的误报。我们在四个最受欢迎且积极维护的DL库(PyTorch、TensorFlow、JAX和OneFlow)上进行了广泛研究。结果表明,$\ abla$Fuzz在代码覆盖率和缺陷检测方面均显著优于最先进的模糊测试器。截至目前,$\ abla$Fuzz为所研究的DL库检测到了173个缺陷,其中144个已被开发者确认(117个是此前未知的缺陷,107个与AD相关)。值得注意的是,在两个月期间,$\ abla$Fuzz贡献了PyTorch和JAX所有高优先级AD缺陷的58.3%(7/12)。现有的模糊测试器均未检测到这些已确认的AD缺陷。

英文摘要:

Deep learning (DL) has attracted wide attention and has been widely deployed in recent years. As a result, more and more research efforts have been dedicated to testing DL libraries and frameworks. However, existing work largely overlooked one crucial component of any DL system, automatic differentiation (AD), which is the basis for the recent development of DL. To this end, we propose $\nabla$Fuzz, the first general and practical approach specifically targeting the critical AD component in DL libraries. Our key insight is that each DL library API can be abstracted into a function processing tensors/vectors, which can be differentially tested under various execution scenarios (for computing outputs/gradients with different implementations). We have implemented $\nabla$Fuzz as a fully automated API-level fuzzer targeting AD in DL libraries, which utilizes differential testing on different execution scenarios to test both first-order and high-order gradients, and also includes automated filtering strategies to remove false positives caused by numerical instability. We have performed an extensive study on four of the most popular and actively-maintained DL libraries, PyTorch, TensorFlow, JAX, and OneFlow. The result shows that $\nabla$Fuzz substantially outperforms state-of-the-art fuzzers in terms of both code coverage and bug detection. To date, $\nabla$Fuzz has detected 173 bugs for the studied DL libraries, with 144 already confirmed by developers (117 of which are previously unknown bugs and 107 are related to AD). Remarkably, $\nabla$Fuzz contributed 58.3% (7/12) of all high-priority AD bugs for PyTorch and JAX during a two-month period. None of the confirmed AD bugs were detected by existing fuzzers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑