arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.09511cs.ARcs.LG
收藏

TrainDeeploy:在极端边缘端实现小规模Transformer模型的硬件加速参数高效微调

TrainDeeploy: Hardware-Accelerated Parameter-Efficient Fine-Tuning of Small Transformer Models at the Extreme Edge

Run Wang, Victor J. B. Jung, Philip Wiese, Francesco Conti, Alessio Burrello, Luca Benini

AI总结:

TrainDeeploy是一种在极端边缘端实现小规模Transformer模型参数高效微调的框架,通过硬件加速和多种训练策略提升训练效率。

摘要:

在设备上对深度神经网络进行微调可以在边缘端实现长期适应,同时保护数据隐私。然而,反向传播的高计算和内存需求对超低功耗、内存受限的极端边缘设备构成了重大挑战。这些挑战在基于注意力的模型中进一步加剧,因为它们的架构复杂性和计算规模较大。我们提出了TrainDeeploy框架,该框架统一了在异构超低功耗系统级芯片(SoCs)上的高效推理和设备端训练。TrainDeeploy为极端边缘SoC提供了完整的设备端训练流水线,支持卷积神经网络(CNNs)和Transformer模型,并提供了多种训练策略,如选择性层间微调和低秩适应(LoRA)。在基于RISC-V的异构SoC上,我们展示了对紧凑型卷积Transformer(CCT)的首次端到端设备端微调,每秒达到11张训练图像。我们表明,LoRA将动态内存使用减少了23%,将可训练参数和梯度数量减少了15倍,并将内存传输体积减少了1.6倍。TrainDeeploy在CCT(0.28M参数,71-126M FLOPs)上达到每周期4.6 FLOP,在Deep-AE(0.27M参数,0.8M FLOPs)上达到每周期13.4 FLOP,同时扩展了先前框架的范围,以支持在极端边缘平台上进行CNN和Transformer模型的参数高效微调。

英文摘要:

On-device tuning of deep neural networks enables long-term adaptation at the edge while preserving data privacy. However, the high computational and memory demands of backpropagation pose significant challenges for ultra-low-power, memory-constrained extreme-edge devices. These challenges are further amplified for attention-based models due to their architectural complexity and computational scale. We present TrainDeeploy, a framework that unifies efficient inference and on-device training on heterogeneous ultra-low-power System-on-Chips (SoCs). TrainDeeploy provides the first complete on-device training pipeline for extreme-edge SoCs supporting both Convolutional Neural Networks (CNNs) and Transformer models, together with multiple training strategies such as selective layer-wise fine-tuning and Low-Rank Adaptation (LoRA). On a RISC-V-based heterogeneous SoC, we demonstrate the first end-to-end on-device fine-tuning of a Compact Convolutional Transformer (CCT), achieving up to 11 trained images per second. We show that LoRA reduces dynamic memory usage by 23%, decreases the number of trainable parameters and gradients by 15x, and reduces memory transfer volume by 1.6x compared to full backpropagation. TrainDeeploy achieves up to 4.6 FLOP/cycle on CCT (0.28M parameters, 71-126M FLOPs) and up to 13.4 FLOP/cycle on Deep-AE (0.27M parameters, 0.8M FLOPs), while expanding the scope of prior frameworks to support both CNN and Transformer models with parameter-efficient tuning on extreme-edge platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑