嵌入式系统设备上深度网络推理的隐性成本
Hidden costs for inference with deep network on embedded system devices
AI总结:
本研究揭示了嵌入式系统中深度网络推理的隐性成本,指出乘法-累加操作指标的局限性,并通过实验强调了张量间额外计算在优化实时性能中的重要性。
摘要:
本研究评估了各种深度学习模型在嵌入式系统环境下的推理性能。在以往的工作中,通常使用乘法-累加操作来衡量深度模型的计算负载。然而,根据本研究,这一指标在估计嵌入式设备上的推理时间时存在局限性。本文提出了一个问题:在以乘法-累加操作为单位表达时,哪些方面被忽略了?在实验中,使用CIFAR-100数据集在嵌入式系统设备上执行图像分类任务,比较和分析了十个深度模型的推理时间与每个模型理论计算的乘法-累加操作数。结果强调了在为嵌入式系统实时性能优化深度学习模型时,考虑张量之间的额外计算的重要性。
英文摘要:
This study evaluates the inference performance of various deep learning models under an embedded system environment. In previous works, Multiply-Accumulate operation is typically used to measure computational load of a deep model. According to this study, however, this metric has a limitation to estimate inference time on embedded devices. This paper poses the question of what aspects are overlooked when expressed in terms of Multiply-Accumulate operations. In experiments, an image classification task is performed on an embedded system device using the CIFAR-100 dataset to compare and analyze the inference times of ten deep models with the theoretically calculated Multiply-Accumulate operations for each model. The results highlight the importance of considering additional computations between tensors when optimizing deep learning models for real-time performing in embedded systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。