arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.01698cs.CCcs.LG
收藏

嵌入式系统设备上深度网络推理的隐性成本

Hidden costs for inference with deep network on embedded system devices

Department of Electronics Engineering Gangneung-Wonju National University(电子工程系,江陵-Wonju国立大学);Department of Advanced AI LG Electronics(先进人工智能系,LG电子)

Chankyu Lee, Woohyun Choi, Sangwook Park

AI总结:

本研究揭示了嵌入式系统中深度网络推理的隐性成本,指出乘法-累加操作指标的局限性,并通过实验强调了张量间额外计算在优化实时性能中的重要性。

摘要:

本研究评估了各种深度学习模型在嵌入式系统环境下的推理性能。在以往的工作中,通常使用乘法-累加操作来衡量深度模型的计算负载。然而,根据本研究,这一指标在估计嵌入式设备上的推理时间时存在局限性。本文提出了一个问题:在以乘法-累加操作为单位表达时,哪些方面被忽略了?在实验中,使用CIFAR-100数据集在嵌入式系统设备上执行图像分类任务,比较和分析了十个深度模型的推理时间与每个模型理论计算的乘法-累加操作数。结果强调了在为嵌入式系统实时性能优化深度学习模型时,考虑张量之间的额外计算的重要性。

英文摘要:

This study evaluates the inference performance of various deep learning models under an embedded system environment. In previous works, Multiply-Accumulate operation is typically used to measure computational load of a deep model. According to this study, however, this metric has a limitation to estimate inference time on embedded devices. This paper poses the question of what aspects are overlooked when expressed in terms of Multiply-Accumulate operations. In experiments, an image classification task is performed on an embedded system device using the CIFAR-100 dataset to compare and analyze the inference times of ten deep models with the theoretically calculated Multiply-Accumulate operations for each model. The results highlight the importance of considering additional computations between tensors when optimizing deep learning models for real-time performing in embedded systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑