arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.11285cs.LGcs.CVcs.DC
收藏

MAS:面向资源高效的联邦多任务学习

MAS: Towards Resource-Efficient Federated Multiple-Task Learning

Nanyang Technological University(南洋理工大学);Sony AI(索尼人工智能);SenseTime Research(商汤科技研究院)

Weiming Zhuang, Yonggang Wen, Lingjuan Lyu, Shuai Zhang

AI总结:

针对多联邦学习任务同时运行易导致边缘设备资源过载的问题,提出MAS(合并与拆分)方法,通过任务合并训练再按亲和度拆分续训的方式,在提升性能的同时将训练时间减半、能耗降低40%。

摘要:

联邦学习(FL)是一种新兴的分布式机器学习方法,支持在去中心化边缘设备上进行原位模型训练。然而,多个同时运行的FL任务可能会让资源受限的设备过载。在本研究中,我们提出了首个能有效协调并训练多个同时运行的FL任务的FL系统。我们首先对同时训练多个FL任务的问题进行了形式化定义。随后,我们提出了新方法MAS(Merge and Split,合并与拆分),以优化多个同时运行的FL任务的训练性能。MAS首先将多个FL任务合并为一个采用多任务架构的一体化FL任务。经过数轮训练后,MAS利用一体化训练期间测得的任务间亲和度,将该一体化FL任务拆分为两个或更多FL任务。之后,它基于一体化训练得到的模型参数,继续训练每个拆分后的FL任务组。大量实验表明,MAS性能优于其他方法,同时将训练时间缩短了2倍,能耗降低了40%。我们希望本研究能启发学界进一步研究和优化同时训练多个FL任务的问题。

英文摘要:

Federated learning (FL) is an emerging distributed machine learning method that empowers in-situ model training on decentralized edge devices. However, multiple simultaneous FL tasks could overload resource-constrained devices. In this work, we propose the first FL system to effectively coordinate and train multiple simultaneous FL tasks. We first formalize the problem of training simultaneous FL tasks. Then, we present our new approach, MAS (Merge and Split), to optimize the performance of training multiple simultaneous FL tasks. MAS starts by merging FL tasks into an all-in-one FL task with a multi-task architecture. After training for a few rounds, MAS splits the all-in-one FL task into two or more FL tasks by using the affinities among tasks measured during the all-in-one training. It then continues training each split of FL tasks based on model parameters from the all-in-one training. Extensive experiments demonstrate that MAS outperforms other methods while reducing training time by 2x and reducing energy consumption by 40%. We hope this work will inspire the community to further study and optimize training simultaneous FL tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑