用于并行分布式推理的深度模型的重构、修剪和调整
Restructuring, Pruning, and Adjustment of Deep Models for Parallel Distributed Inference
摘要：
使用多个节点和并行计算算法已经成为提高深度神经网络的训练和执行时间以及传感器网络中的有效集体智能的主要工具。在本文中，我们考虑在多个处理节点上并行实现一个已经训练好的深度模型（也称为worker），其中深度模型被划分为若干并行子模型，每个子模型由worker执行。由于工作者之间的同步和数据传输引起的延迟会对并行实现的性能产生负面影响，因此希望并行子模型之间的相互依赖性最小。为了实现这一目标，我们建议重新安排神经网络中的神经元，并将它们分区（而不改变神经网络的一般拓扑结构），使得子模型之间的相互依赖性最小化的计算和通信约束下的工人。我们提出了RePurpose，逐层模型重构和修剪技术，保证了整体并行模型的性能。为了有效地应用RePurpose，我们提出了一种基于$\ell_0$优化和Munkres分配算法的方法。我们表明，与现有的方法相比，RePurpose显着提高了效率的分布式推理通过并行实现，无论是在通信和计算复杂性。
英文摘要：
Using multiple nodes and parallel computing algorithms has become a principal tool to improve training and execution times of deep neural networks as well as effective collective intelligence in sensor networks. In this paper, we consider the parallel implementation of an already-trained deep model on multiple processing nodes (a.k.a. workers) where the deep model is divided into several parallel sub-models, each of which is executed by a worker. Since latency due to synchronization and data transfer among workers negatively impacts the performance of the parallel implementation, it is desirable to have minimum interdependency among parallel sub-models. To achieve this goal, we propose to rearrange the neurons in the neural network and partition them (without changing the general topology of the neural network), such that the interdependency among sub-models is minimized under the computations and communications constraints of the workers. We propose RePurpose, a layer-wise model restructuring and pruning technique that guarantees the performance of the overall parallelized model. To efficiently apply RePurpose, we propose an approach based on $\ell_0$ optimization and the Munkres assignment algorithm. We show that, compared to the existing methods, RePurpose significantly improves the efficiency of the distributed inference via parallel implementation, both in terms of communication and computational complexity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
