基于非穿透式张量分区（Non-Penetrative Tensor Partitioning）的协同推理加速
Collaborative Inference Acceleration with Non-Penetrative Tensor Partitioning
AI总结：
针对IoT设备协同DNN推理中图块共享通信开销高的问题，提出细粒度NPTP张量分区方法以最小化通信负载，并在四种DNN上实现相对CoEdge 1.44–1.68倍加速。
摘要：
在物联网（IoT）设备上进行大尺寸图像推理通常受到有限资源的限制，同时深度神经网络（DNN）推理往往存在严格的延迟要求。目前，这一问题通常通过协同推理来解决：将大尺寸图像划分为多个图块，并将每个图块分配给一台IoT设备进行处理。然而，由于图块共享所导致的通信开销会带来显著延迟，现有协同推理策略对于卷积计算而言效率低下，而卷积计算对任何DNN都是不可或缺的。为降低该延迟，我们提出非穿透式张量分区（Non-Penetrative Tensor Partitioning，NPTP），这是一种细粒度张量分区方法，通过最小化共享图块的通信负载来降低通信延迟，从而降低推理延迟。我们使用四种被广泛采用的DNN模型评估NPTP。实验结果表明，相对于最先进（SOTA）协同推理算法CoEdge，NPTP实现了1.44–1.68倍的推理加速。
英文摘要：
The inference of large-sized images on Internet of Things (IoT) devices is commonly hindered by limited resources, while there are often stringent latency requirements for Deep Neural Network (DNN) inference. Currently, this problem is generally addressed by collaborative inference, where the large-sized image is partitioned into multiple tiles, and each tile is assigned to an IoT device for processing. However, since significant latency will be incurred due to the communication overhead caused by tile sharing, the existing collaborative inference strategy is inefficient for convolutional computation, which is indispensable for any DNN. To reduce it, we propose Non-Penetrative Tensor Partitioning (NPTP), a fine-grained tensor partitioning method that reduces the communication latency by minimizing the communication load of tiles shared, thereby reducing inference latency. We evaluate NPTP with four widely-adopted DNN models. Experimental results demonstrate that NPTP achieves a 1.44-1.68x inference speedup relative to CoEdge, a state-of-the-art (SOTA) collaborative inference algorithm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
