模型分区能否提升联邦学习的可持续性?
Could Model Partitioning Make Federated Learning More Sustainable?
AI总结:
针对联邦学习的碳足迹问题,提出模型分区技术以转移能耗,初步发现其可降低参与者能耗达76%且无显著额外开销。
AI中文摘要:
随着联邦学习(FL)从低功耗设备间的分布式机器学习扩展到涉及边缘服务器和数据中心的跨筒仓场景,其碳足迹已成为日益突出的问题。为解决该问题,可持续联邦学习方法会使训练与低碳能源供应或低电网需求相匹配,并通过减小模型规模降低由高碳能源供电的客户端的能耗。我们提出应用模型分区技术,该技术可根据碳感知或电网感知信号,通过将模型的部分计算任务卸载给其他参与者来转移能耗。初步研究结果表明,对于某些分区点,与未分区的训练相比,模型分区可将参与者的能耗降低多达76%,且不会产生显著的时间或能耗开销。
英文摘要:
As federated learning (FL) extends from distributed machine learning between low-power devices to cross-silo scenarios involving edge servers and data centres, its carbon footprint has become a growing concern. Addressing this, methods for sustainable FL align training with low-carbon energy availability or low grid demand and reduce the energy consumption of clients powered by high-carbon sources by decreasing the size of their models. We propose applying model partitioning, which can shift energy consumption by offloading parts of a model to another participant, in response to carbon- or grid-aware signals. Our preliminary findings show that for some partition points, model partitioning can reduce a participant's energy consumption by up to 76% without any significant time or energy consumption overhead compared to non-partitioned training.