arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.07957cs.LGcs.AI
收藏

面向异构和分布式物联网环境的自适应推理高效训练后增强方法

Efficient Post-Training Augmentation for Adaptive Inference in Heterogeneous and Distributed IoT Environments

Infineon Technologies Dresden GmbH & Co. KG(英飞凌技术德累斯顿有限合伙公司);TU Dresden(德累斯顿工业大学);TU Munich(慕尼黑工业大学);Infineon Technologies AG(英飞凌技术股份公司)

Max Sponner, Lorenzo Servadei, Bernd Waschneck, Robert Wille, Akash Kumar

AI总结:

提出自动化训练后增强流程,将现有模型转换为早期退出神经网络,自动完成架构构建、子图映射和决策配置,在物联网任务上显著降低推理计算与能耗,且搜索成本低。

摘要:

早期退出神经网络(EENN)为提升神经网络部署效率提供了一种解决方案。然而,由于存在大量额外的设计选择,创建EENN具有挑战性,且需要专门的领域知识。为解决这一问题,我们提出了一种自动化的增强流程,专注于将现有模型转换为EENN。该流程针对异构或分布式硬件目标的部署执行所有必要的设计决策:我们的框架构建EENN架构,将其子图映射到硬件目标,并配置其决策机制。据我们所知,这是第一个能够执行所有这些步骤的框架。\n我们在一组物联网和标准图像分类用例上评估了我们的方法。对于语音命令检测任务,我们的解决方案能够将每次推理的平均操作数减少59.67%。对于心电图(ECG)分类任务,它能够提前终止所有样本,将平均推理能耗降低74.9%,计算量减少78.3%。在CIFAR-10上,我们的解决方案能够实现高达58.75%的计算量减少。\n在笔记本电脑CPU上,针对CIFAR-10的ResNet-152基础模型的搜索耗时不到九小时。我们提出的方法能够创建针对物联网环境优化的EENN,并降低嵌入式平台和雾计算平台上深度学习应用的推理成本,同时显著降低搜索成本——使其对工业界和科研领域的科学家和工程师更加易用。低搜索成本提高了EENN的可访问性,有潜力在广泛的实际应用中提升神经网络的效率。

英文摘要:

Early Exit Neural Networks (EENNs) present a solution to enhance the efficiency of neural network deployments. However, creating EENNs is challenging and requires specialized domain knowledge, due to the large amount of additional design choices. To address this issue, we propose an automated augmentation flow that focuses on converting an existing model into an EENN. It performs all required design decisions for the deployment to heterogeneous or distributed hardware targets: Our framework constructs the EENN architecture, maps its subgraphs to the hardware targets, and configures its decision mechanism. To the best of our knowledge, it is the first framework that is able to perform all of these steps. We evaluated our approach on a collection of Internet-of-Things and standard image classification use cases. For a speech command detection task, our solution was able to reduce the mean operations per inference by 59.67%. For an ECG classification task, it was able to terminate all samples early, reducing the mean inference energy by 74.9% and computations by 78.3%. On CIFAR-10, our solution was able to achieve up to a 58.75% reduction in computations. The search on a ResNet-152 base model for CIFAR-10 took less than nine hours on a laptop CPU. Our proposed approach enables the creation of EENN optimized for IoT environments and can reduce the inference cost of Deep Learning applications on embedded and fog platforms, while also significantly reducing the search cost - making it more accessible for scientists and engineers in industry and research. The low search cost improves the accessibility of EENNs, with the potential to improve the efficiency of neural networks in a wide range of practical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑