arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.05201cs.LGcs.AIcs.DC
收藏

TinyReptile:基于联邦元学习的TinyML

TinyReptile: TinyML with Federated Meta-Learning

Siemens AG(西门子股份公司);Technical University of Munich(慕尼黑工业大学)

Haoyu Ren, Darko Anicic, Thomas A. Runkler

AI总结:

本文提出TinyReptile算法,结合联邦元学习与在线学习,在资源受限的微型设备上协作学习神经网络初始化,实现快速适配新设备,并在Raspberry Pi 4和Cortex-M4 MCU上验证了资源与训练时间至少减少两倍的效果。

摘要:

微型机器学习(TinyML)是一个快速发展的领域,旨在使机器学习(ML)普及到资源受限的微控制器(MCU)上。鉴于这些微型设备的普遍性,一个自然的问题是:TinyML应用能否从聚合各自的知识中获益。联邦学习(FL)使分散的智能体能够在不上传敏感本地数据的情况下联合学习一个全局模型。然而,由于实际部署环境的复杂性以及每个设备上可用数据的异构性,一个通用的全局模型可能并不适用于所有设备。此外,TinyML硬件的部署具有显著的计算和通信约束,传统ML无法解决这些问题。考虑到这些挑战,我们提出了TinyReptile,这是一种受元学习和在线学习启发的简单而高效的算法,用于在微型设备之间协作学习神经网络(NN)的可靠初始化,使其能够针对新设备的数据快速适配。我们在Raspberry Pi 4和仅有256-KB RAM的Cortex-M4 MCU上演示了TinyReptile。在多种TinyML用例上的评估表明,与性能相当的基线算法相比,资源消耗和训练时间至少减少了两倍。

英文摘要:

Tiny machine learning (TinyML) is a rapidly growing field aiming to democratize machine learning (ML) for resource-constrained microcontrollers (MCUs). Given the pervasiveness of these tiny devices, it is inherent to ask whether TinyML applications can benefit from aggregating their knowledge. Federated learning (FL) enables decentralized agents to jointly learn a global model without sharing sensitive local data. However, a common global model may not work for all devices due to the complexity of the actual deployment environment and the heterogeneity of the data available on each device. In addition, the deployment of TinyML hardware has significant computational and communication constraints, which traditional ML fails to address. Considering these challenges, we propose TinyReptile, a simple but efficient algorithm inspired by meta-learning and online learning, to collaboratively learn a solid initialization for a neural network (NN) across tiny devices that can be quickly adapted to a new device with respect to its data. We demonstrate TinyReptile on Raspberry Pi 4 and Cortex-M4 MCU with only 256-KB RAM. The evaluations on various TinyML use cases confirm a resource reduction and training time saving by at least two factors compared with baseline algorithms with comparable performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑