工业机器视觉中深度学习模型的边缘推理策略基准测试
Benchmarking Edge Inference Strategies for Deep Learning Models in Industrial Machine Vision
浏览论文内容
中文总结 AI 辅助
研究工业机器视觉中深度学习模型的边缘推理策略，比较普通PyTorch、ONNX Runtime、OpenVINO和TensorRT四种方法，评估其在多硬件平台及不同模型下的推理时间，得出不同平台上各方法的性能表现。
中文摘要 AI 辅助
当低延迟、数据安全或有限连接是关键要求时，边缘部署通常是工业机器视觉系统的首选解决方案。有几种框架可用于优化边缘设备上的推理，但相对较少的研究在工业部署条件下系统地比较它们的推理时间性能。在这项工作中，我们对工业环境中机器视觉推理的四种广泛使用的方法进行了比较研究：普通PyTorch、ONNX Runtime、OpenVINO和TensorRT。评估集中在推理时间上，涵盖了几个基于CPU和GPU的硬件平台，包括传统卷积神经网络和基于Transformer的视觉模型。对于评估的硬件平台和模型，结果表明OpenVINO在CPU上实现了最低推理时间，而TensorRT在GPU上实现了最低推理时间。然而，对于本研究中考虑的基于Transformer的模型，TensorRT并不优于普通PyTorch。
英文摘要
Edge deployment is often the preferred solution for industrial machine vision systems when low latency, data security, or limited connectivity are critical requirements. Several frameworks are available to optimise inference on edge devices; however, relatively few studies have systematically compared their inference-time performance under industrial deployment conditions. In this work, we present a comparative study of four widely used approaches for machine vision inference in industrial settings: plain PyTorch, ONNX Runtime, OpenVINO, and TensorRT. The evaluation focuses on inference time, covers several CPU- and GPU-based hardware platforms, and includes both conventional convolutional neural networks and a transformer-based vision model. For the hardware platforms and models evaluated, the results show that OpenVINO achieves the lowest inference time on CPUs, while TensorRT achieves the lowest inference time on GPUs. However, TensorRT does not outperform plain PyTorch for the transformer-based model considered in this study.
发表机构
- Spanish Ministry of Science, Innovation and Universities（西班牙科学、创新与大学部）
- Spanish State Research Agency（西班牙国家研究机构）
- European Regional Development Fund（欧洲区域发展基金）
- European Union（欧洲联盟）
- European Health and Digital Executive Agency（欧洲健康与数字执行机构）
机构由 AI 辅助整理，请以论文原文为准。
