arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.02551cs.SEcs.AIcs.LG
收藏

预先训练的深度神经网络之间的差异:对模型动物园可靠性的新威胁

Discrepancies among Pre-trained Deep Neural Networks: A New Threat to Model Zoo Reliability

Purdue University(普渡大学)

Diego Montes, Pongpatapee Peerapatanapokin, Jeff Schultz, Chengjun Gun, Wenxin Jiang, James C. Davis

摘要:

训练深度神经网络(DNN)需要大量的时间和资源。加快部署的一种做法是使用预先训练的深度神经网络(PTNN),通常来自模型动物园-PTNN的集合;然而,模型动物园的可靠性仍然未经检验。在没有PTNN实现和性能的行业标准的情况下,工程师无法自信地将其纳入生产系统。作为第一步,发现模型动物园中PTNN之间的潜在差异将揭示对模型动物园可靠性的威胁。先前的工作表明深度学习系统在准确性方面存在差异。然而,模型动物园PTNN的可靠性的更广泛的措施是未经探索的。这项工作测量了四个模型动物园中36个PTNN的准确性,延迟和架构之间的显着差异。在前10个差异中,我们发现准确率差异为1.23%-2.62%,延迟差异为9%-131%。我们还发现了众所周知的DNN架构的架构不匹配(例如,ResNet和AlexNet)。我们的研究结果呼吁未来的工作经验验证,自动化工具的测量,并实施的最佳实践。

英文摘要:

Training deep neural networks (DNNs) takes signifcant time and resources. A practice for expedited deployment is to use pre-trained deep neural networks (PTNNs), often from model zoos -- collections of PTNNs; yet, the reliability of model zoos remains unexamined. In the absence of an industry standard for the implementation and performance of PTNNs, engineers cannot confidently incorporate them into production systems. As a first step, discovering potential discrepancies between PTNNs across model zoos would reveal a threat to model zoo reliability. Prior works indicated existing variances in deep learning systems in terms of accuracy. However, broader measures of reliability for PTNNs from model zoos are unexplored. This work measures notable discrepancies between accuracy, latency, and architecture of 36 PTNNs across four model zoos. Among the top 10 discrepancies, we find differences of 1.23%-2.62% in accuracy and 9%-131% in latency. We also fnd mismatches in architecture for well-known DNN architectures (e.g., ResNet and AlexNet). Our findings call for future works on empirical validation, automated tools for measurement, and best practices for implementation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑