压缩还是不压缩：表征嵌入式推理的深度学习模型压缩
To Compress, or Not to Compress: Characterizing Deep Learning Model Compression for Embedded Inference
摘要：
深度神经网络（DNN）的最新进展使其对嵌入式系统具有吸引力。然而，DNN可能需要很长时间才能在资源受限的计算设备上进行推断。模型压缩技术可以解决嵌入式设备上深度推理的计算问题。这种技术非常有吸引力，因为它不依赖于专用硬件，也不依赖于由于隐私问题或高延迟而通常不可行的计算卸载。然而，目前还不清楚模型压缩技术如何在各种DNN上执行。为了设计高效的嵌入式深度学习解决方案，我们需要了解它们的行为。这项工作开发了一种定量方法来表征模型压缩技术的代表性嵌入式深度学习架构，NVIDIA Jetson Tx2。我们进行了广泛的实验，考虑从图像分类和自然语言处理域的11个有影响力的神经网络架构。我们的实验表明，如何两个主流的压缩技术，数据量化和修剪，执行这些网络架构和压缩技术的影响模型的存储大小，推理时间，能耗和性能指标。我们证明了有机会在嵌入式系统上实现快速的深度推理，但必须仔细选择压缩设置。我们的结果提供了有关何时以及如何应用模型压缩技术的见解，以及设计高效嵌入式深度学习系统的指南。
英文摘要：
The recent advances in deep neural networks (DNNs) make them attractive for embedded systems. However, it can take a long time for DNNs to make an inference on resource-constrained computing devices. Model compression techniques can address the computation issue of deep inference on embedded devices. This technique is highly attractive, as it does not rely on specialized hardware, or computation-offloading that is often infeasible due to privacy concerns or high latency. However, it remains unclear how model compression techniques perform across a wide range of DNNs. To design efficient embedded deep learning solutions, we need to understand their behaviors. This work develops a quantitative approach to characterize model compression techniques on a representative embedded deep learning architecture, the NVIDIA Jetson Tx2. We perform extensive experiments by considering 11 influential neural network architectures from the image classification and the natural language processing domains. We experimentally show that how two mainstream compression techniques, data quantization and pruning, perform on these network architectures and the implications of compression techniques to the model storage size, inference time, energy consumption and performance metrics. We demonstrate that there are opportunities to achieve fast deep inference on embedded systems, but one must carefully choose the compression settings. Our results provide insights on when and how to apply model compression techniques and guidelines for designing efficient embedded deep learning systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
