面向深度学习的协作智能友好架构
Towards Collaborative Intelligence Friendly Architectures for Deep Learning
摘要：
现代移动设备配备了图形处理器（GPU）等高性能硬件资源，使端侧智能服务变得更加可行。即使是最近，专门的硅作为神经引擎也被用于移动设备。然而，大多数移动设备仍然无法使用非常深入的模型执行实时推理。与当今智能应用的深度模型相关的计算通常仅在云上执行。这种仅云的方法需要通过移动无线网络将大量原始数据上传到云，并在云服务器上施加相当大的计算和通信负载。最近的研究表明，通过在移动终端和云端之间分担工作负载，可以显著降低移动应用中深度神经网络的延迟和能耗。在这种称为协作智能的方法中，在移动终端上计算的中间特征被卸载到云，而不是网络的原始输入数据，从而减少了需要发送到云的数据的大小。在本文中，我们通过引入一个负责更大程度地减少需要卸载到云的特征数据大小的单元，设计了一种新的协作智能友好架构，其中该单元放置在深度模型的选定层之后。我们提出的方法在不同的无线网络中，与ResNet-50的现状仅云方法相比，端到端延迟平均提高了53倍，移动能耗平均提高了68倍，而准确性损失不到2%。
英文摘要：
Modern mobile devices are equipped with high-performance hardware resources such as graphics processing units (GPUs), making the end-side intelligent services more feasible. Even recently, specialized silicons as neural engines are being used for mobile devices. However, most mobile devices are still not capable of performing real-time inference using very deep models. Computations associated with deep models for today's intelligent applications are typically performed solely on the cloud. This cloud-only approach requires significant amounts of raw data to be uploaded to the cloud over the mobile wireless network and imposes considerable computational and communication load on the cloud server. Recent studies have shown that the latency and energy consumption of deep neural networks in mobile applications can be notably reduced by splitting the workload between the mobile device and the cloud. In this approach, referred to as collaborative intelligence, intermediate features computed on the mobile device are offloaded to the cloud instead of the raw input data of the network, reducing the size of the data needed to be sent to the cloud. In this paper, we design a new collaborative intelligence friendly architecture by introducing a unit responsible for reducing the size of the feature data needed to be offloaded to the cloud to a greater extent, where this unit is placed after a selected layer of a deep model. Our proposed method, across different wireless networks, achieves on average 53x improvements for end-to-end latency and 68x improvements for mobile energy consumption compared to the status quo cloud-only approach for ResNet-50, while the accuracy loss is less than 2%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
