arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.03402cs.LGcs.AI
收藏

通过多精度空中聚合实现混合精度联邦学习

Mixed-Precision Federated Learning via Multi-Precision Over-The-Air Aggregation

Faculty of Engineering & Applied Sciences(工程与应用科学学院);Department of Engineering(工程系);Cranfield University(克兰菲尔德大学);Durham University(达勒姆大学)

Jinsheng Yuan, Zhuangkun Wei, Weisi Guo

AI总结:

针对异构客户端不同比特精度导致的空中聚合不兼容问题,提出多精度梯度调制的混合精度 OTA-FL 框架,在真实数据上验证其精度与能耗优势。

摘要:

空中联邦学习(Over-the-Air Federated Learning,OTA-FL)是一种隐私保护的分布式学习机制,它在电磁信道中而非服务器端聚合更新。现有 OTA-FL 研究中的一个关键空白是假设客户端计算比特精度同质。而在实际应用中,具有不同硬件资源的客户端可能利用近似计算(approximate computing,AxC),以针对能耗和计算效率优化的不同比特精度运行。客户端之间各种精度的模型更新给 OTA-FL 带来了一个开放性挑战,因为它与无线调制叠加不兼容。在此,我们提出一种面向具有多种比特精度客户端的混合精度 OTA-FL 框架,并展示以下创新:(i)与同质客户端比特精度相比,在边缘计算能力各异、能效和学习精度要求的约束下,为服务器和客户端都实现更优的权衡;(ii)一种多精度梯度调制方案,以确保与 OTA 聚合兼容并消除精度转换开销。通过使用真实世界数据进行案例研究,我们验证了该调制方案能够实现基于 AxC 的混合精度 OTA-FL。与 32 位和 16 位的同质标准精度相比,我们的框架在 4 位超低精度客户端性能方面提升超过 10%,并分别实现超过 65% 和 13% 的能耗节省。这证明了我们的混合精度 OTA-FL 方法在异构边缘计算环境中的巨大潜力。

英文摘要:

Over-the-Air Federated Learning (OTA-FL) is a privacy-preserving distributed learning mechanism, by aggregating updates in the electromagnetic channel rather than at the server. A critical research gap in existing OTA-FL research is the assumption of homogeneous client computational bit precision. While in real world application, clients with varying hardware resources may exploit approximate computing (AxC) to operate at different bit precisions optimized for energy and computational efficiency. And model updates of various precisions amongst clients poses an open challenge for OTA-FL, as it is incompatible in the wireless modulation superposition. Here, we propose an mixed-precision OTA-FL framework of clients with multiple bit precisions, demonstrating the following innovations: (i) the superior trade-off for both server and clients within the constraints of varying edge computing capabilities, energy efficiency, and learning accuracy requirements comparing to homogeneous client bit precision, and (ii) a multi-precision gradient modulation scheme to ensure compatibility with OTA aggregation and eliminate the overheads of precision conversion. Through case study with real world data, we validate our modulation scheme that enables AxC based mixed-precision OTA-FL. In comparison to homogeneous standard precision of 32-bit and 16-bit, our framework presents more than 10% in 4-bit ultra low precision client performance and over 65%and 13% of energy savings respectively. This demonstrates the great potential of our mixed-precision OTA-FL approach in heterogeneous edge computing environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑