arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.14628cs.SEcs.LG
收藏

从深度学习模型推断数据前置条件以实现部署中的可信预测

Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment

Iowa State University(爱荷华州立大学)

Shibbir Ahmed, Hongyang Gao, Hridesh Rajan

AI总结:

针对部署时深度神经网络预测可信度判断难题,提出DeepInfer方法,通过新型抽象与最弱前置条件推理推断数据前置条件,在29个真实模型上验证了其高召回率、高F-1分数及更优的运行效率。

摘要:

深度学习模型在开发阶段基于对数据的特定假设进行训练,随后在部署阶段用于预测。在部署阶段,判断模型对未见过数据的预测可信度至关重要。现有的传统软件规约与验证方法不足以完成这项任务,因为它们无法处理DNN(深度神经网络)模型架构的复杂性和预期输出。在本研究中,我们提出了一种新技术,利用从神经网络计算中推导的规则来推断DNN模型的数据前置条件,以判断其预测的可信度。我们的方法DeepInfer引入了一种针对已训练DNN模型的新型抽象机制,该机制可利用Dijkstra谓词转换器语义进行最弱前置条件推理。通过在神经网络抽象表示的归纳类型上推导规则,我们能够解决从输出层到输入层的反向非线性计算所产生的矩阵维度问题。我们利用各类激活函数的规则进行最弱前置条件计算,从深度神经网络最终输出上的给定后置条件出发,逐层计算前置条件。我们使用从五个不同来源收集的四个不同数据集,在29个真实世界DNN模型上对DeepInfer进行了广泛评估,证明了其效用、有效性以及相较于相近工作的性能提升。DeepInfer能高效检测高精度模型的正确和错误预测,召回率达0.98,F-1分数达0.84,相较现有技术SelfChecker有显著提升。DeepInfer的平均运行时开销很低,对所有未见过的数据集仅为0.22秒。我们还在相同硬件设置下比较了运行时开销,发现DeepInfer的速度是SelfChecker的3.27倍。

英文摘要:

Deep learning models are trained with certain assumptions about the data during the development stage and then used for prediction in the deployment stage. It is important to reason about the trustworthiness of the model's predictions with unseen data during deployment. Existing methods for specifying and verifying traditional software are insufficient for this task, as they cannot handle the complexity of DNN model architecture and expected outcomes. In this work, we propose a novel technique that uses rules derived from neural network computations to infer data preconditions for a DNN model to determine the trustworthiness of its predictions. Our approach, DeepInfer involves introducing a novel abstraction for a trained DNN model that enables weakest precondition reasoning using Dijkstra's Predicate Transformer Semantics. By deriving rules over the inductive type of neural network abstract representation, we can overcome the matrix dimensionality issues that arise from the backward non-linear computation from the output layer to the input layer. We utilize the weakest precondition computation using rules of each kind of activation function to compute layer-wise precondition from the given postcondition on the final output of a deep neural network. We extensively evaluated DeepInfer on 29 real-world DNN models using four different datasets collected from five different sources and demonstrated the utility, effectiveness, and performance improvement over closely related work. DeepInfer efficiently detects correct and incorrect predictions of high-accuracy models with high recall (0.98) and high F-1 score (0.84) and has significantly improved over prior technique, SelfChecker. The average runtime overhead of DeepInfer is low, 0.22 sec for all unseen datasets. We also compared runtime overhead using the same hardware settings and found that DeepInfer is 3.27 times faster than SelfChecker.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑