利用边缘人工智能推理加速器实现设备端模型自适应
Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator
浏览论文内容
中文总结 AI 辅助
研究针对资源受限硬件上设备端模型自适应难题，提出用异构自适应管道，借助Hailo-8L加速器提取冻结主干特征，划分计算图，在多架构和数据集上提升训练速度、吞吐量并降低能耗，证明该方法实用。
中文摘要 AI 辅助
设备端模型自适应对于在资源受限硬件上实现终身个性化至关重要，但此类设备的计算、功率和内存限制使现代深度神经网络的端到端反向传播不切实际。本文提出一种异构自适应管道，将商用边缘人工智能推理加速器Hailo-8L用于设备端训练期间的冻结主干特征提取。计算图被划分，预训练主干量化为INT8在加速器上运行，仅在主机CPU上微调轻量级FP32分类头，实现频繁、节能的现场更新且大部分权重保持不变。与树莓派5 CPU基线相比，该管道在多个架构和数据集上实现了高达15.4倍的更快时钟训练时间，在有利设置下提供有竞争力的吞吐量，并持续降低每样本能耗。训练后量化恢复对于保留加速器生成特征的质量和减轻量化敏感架构中的精度损失至关重要。总体而言，结果展示了一种使用面向推理的边缘加速器进行高效设备端自适应的实用方法。
英文摘要
On-device model adaptation is essential to enable lifelong personalization on resource-constrained hardware, but compute, power, and memory limitations of such devices make end-to-end backpropagation impractical for modern deep neural networks. This work proposes a heterogeneous adaptation pipeline that repurposes a commercial edge AI inference accelerator, Hailo-8L, for frozen-backbone feature extraction during on-device training. The computational graph is partitioned so that the pre-trained backbone is quantized to INT8 and run on the accelerator, while only a lightweight FP32 classification head is fine-tuned on the host CPU, enabling frequent, energy-efficient in-field updates with most weights remaining fixed. Across multiple architectures and datasets, this pipeline achieves up to 15.4x faster wall-clock training time compared to a Raspberry Pi 5 CPU baseline, offers competitive throughput in favorable settings, and consistently reduces energy per sample. Post-training quantization restoration is shown to be crucial for preserving the quality of accelerator-generated features and mitigating accuracy loss in quantization-sensitive architectures. Overall, the results demonstrate a practical approach to efficient on-device adaptation using inference-oriented edge accelerators. The implementation is available at https://github.com/MatPiech/accelerator-training.
发表机构
- Poznan University of Technology（波兹南理工大学）
- University of Modena and Reggio Emilia（摩德纳大学和雷焦艾米利亚大学）
机构由 AI 辅助整理，请以论文原文为准。
