arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.17708cs.SEcs.LG
收藏

深度学习模型转换器中的故障与风险分析:以 ONNX 生态系统为例

Analysis of Failures and Risks in Deep Learning Model Converters: A Case Study in the ONNX Ecosystem

Purdue University(普渡大学);Loyola University Chicago(芝加哥洛约拉大学)

Purvish Jajal, Wenxin Jiang, Arav Tewari, Erik Kocinare, Joseph Woo, Anusha Sarraf, Yung-Hsiang Lu, George K. Thiruvathukal, James C. Davis

AI总结:

本文以 ONNX 生态系统为案例,通过开发者调查和 200 个 PyTorch/TensorFlow 转换器问题分析,揭示节点转换阶段约占 75% 缺陷、33% 故障源于语义错误模型,并指出行为不一致模型共享算子序列,为提升 DL 互操作软件的可维护性与验证能力提供依据。

摘要:

软件工程师使用多种开发框架和运行时环境来开发、微调和部署深度学习(DL)模型。DL 模型转换器负责在框架之间以及向运行时环境迁移模型。转换错误会损害模型质量并中断部署。然而,DL 模型转换器的故障特征尚不明确,这增加了使用 DL 互操作技术时的风险。本文分析了 DL 模型转换器中的故障。我们调查了软件工程师关于 DL 互操作工具、用例和痛点的情况(N=92)。随后,我们刻画了与主要互操作工具 ONNX 相关的模型转换器故障特征(N=200 个来自 PyTorch 和 TensorFlow 的问题)。最后,我们针对所研究故障的结构性原因提出并检验了两个假设。我们发现,模型转换器的节点转换阶段约占缺陷的 75%,且 33% 的报告故障与语义不正确的模型有关。语义不正确模型的成因难以捉摸,但行为不一致的模型共享某些算子序列。我们的结果激励了未来研究,以使 DL 互操作软件更易于维护、扩展和验证。对行为容差和架构覆盖率指标的研究可能富有成效。

英文摘要:

Software engineers develop, fine-tune, and deploy deep learning (DL) models using a variety of development frameworks and runtime environments. DL model converters move models between frameworks and to runtime environments. Conversion errors compromise model quality and disrupt deployment. However, the failure characteristics of DL model converters are unknown, adding risk when using DL interoperability technologies. This paper analyzes failures in DL model converters. We survey software engineers about DL interoperability tools, use cases, and pain points (N=92). Then, we characterize failures in model converters associated with the main interoperability tool, ONNX (N=200 issues in PyTorch and TensorFlow). Finally, we formulate and test two hypotheses about structural causes for the failures we studied. We find that the node conversion stage of a model converter accounts for ~75% of the defects and 33% of reported failure are related to semantically incorrect models. The cause of semantically incorrect models is elusive, but models with behaviour inconsistencies share operator sequences. Our results motivate future research on making DL interoperability software simpler to maintain, extend, and validate. Research into behavioural tolerances and architectural coverage metrics could be fruitful.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑