arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.03246cs.LGcs.AI
收藏

MobileTL:基于倒残差块的端侧迁移学习

MobileTL: On-device Transfer Learning with Inverted Residual Blocks

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Hung-Yueh Chiang, Natalia Frumkin, Feng Liang, Diana Marculescu

AI总结:

提出MobileTL方法,通过训练归一化层偏移、近似激活层后向计算及微调顶部块,实现基于倒残差块模型的内存与计算高效端侧迁移学习。

摘要:

由于端侧设备资源有限,边缘迁移学习面临挑战。现有工作通过训练部分参数或添加模型补丁来解决此问题。倒残差块(IRBs)专为推理设计,将卷积层拆分为深度卷积和逐点卷积,导致堆叠层增多(如卷积、归一化和激活层)。尽管IRBs推理高效,但训练卷积层权重和归一化层缩放参数时,需在内存中存储额外激活图。因此,高内存开销阻碍了在资源受限的边缘设备上训练IRBs,使其不适用于迁移学习场景。为解决此问题,我们提出MobileTL,一种针对基于IRBs构建模型的内存高效且计算高效的端侧迁移学习方法。MobileTL训练内部归一化层的偏移参数,避免在后向传播中存储激活图。此外,MobileTL将激活层(如Hard-Swish和ReLU6)的后向计算近似为带符号函数,从而在后向传播中仅需存储二值掩码而非激活图。为降低计算成本,MobileTL微调靠近输出的顶部少量块,而非通过整个网络传播梯度。该方法在MobileNetV2和MobileNetV3的IRBs上分别降低46%和53%的内存使用。对于MobileNetV3,微调5个块时浮点运算(FLOPs)减少36%,在CIFAR10上精度仅下降0.6%。多数据集实验表明,与边缘设备迁移学习现有工作相比,该方法达到帕累托最优(给定硬件约束下精度最佳)。

英文摘要:

Transfer learning on edge is challenging due to on-device limited resources. Existing work addresses this issue by training a subset of parameters or adding model patches. Developed with inference in mind, Inverted Residual Blocks (IRBs) split a convolutional layer into depthwise and pointwise convolutions, leading to more stacking layers, e.g., convolution, normalization, and activation layers. Though they are efficient for inference, IRBs require that additional activation maps are stored in memory for training weights for convolution layers and scales for normalization layers. As a result, their high memory cost prohibits training IRBs on resource-limited edge devices, and making them unsuitable in the context of transfer learning. To address this issue, we present MobileTL, a memory and computationally efficient on-device transfer learning method for models built with IRBs. MobileTL trains the shifts for internal normalization layers to avoid storing activation maps for the backward pass. Also, MobileTL approximates the backward computation of the activation layer (e.g., Hard-Swish and ReLU6) as a signed function which enables storing a binary mask instead of activation maps for the backward pass. MobileTL fine-tunes a few top blocks (close to output) rather than propagating the gradient through the whole network to reduce the computation cost. Our method reduces memory usage by 46% and 53% for MobileNetV2 and V3 IRBs, respectively. For MobileNetV3, we observe a 36% reduction in floating-point operations (FLOPs) when fine-tuning 5 blocks, while only incurring a 0.6% accuracy reduction on CIFAR10. Extensive experiments on multiple datasets demonstrate that our method is Pareto-optimal (best accuracy under given hardware constraints) compared to prior work in transfer learning for edge devices.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑