arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.14751cs.ITcs.DCcs.LGeess.SPmath.IT
收藏

利用联邦学习中的相关性:机遇与实际限制

Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations

Department of Electrical Engineering (ISY), Linköping University(_linköping大学电气工程系);CentraleSupélec, CNRS, L2S, Université Paris-Saclay(_centralesupelec, CNRS, L2S, 巴黎萨克雷大学);Division of Decision and Control Systems, School of EECS, KTH Royal Institute of Technology(_决策与控制系统系,皇家理工学院)

Adrian Edin, Michel Kieffer, Mikael Johansson, Zheng Chen

AI总结:

本文系统分类了梯度和模型压缩方案,探讨了任务复杂度、模型架构对相关性的影响,并提出自适应压缩方法以提升联邦学习效率。

摘要:

联邦学习(FL)中的通信瓶颈推动了减少客户端与中央参数服务器之间数据交换的研究。本文将梯度和模型压缩方案分为结构性、时序性和空间性三类相关性。我们探讨了这些相关性的来源,提出了衡量其强度的量化指标,并通过统一的相关性框架重新解释了现有压缩方法。实验表明,结构性、时序性和空间性相关性的程度取决于任务复杂性、模型架构和算法配置。这些发现表明,算法设计者应在特定部署场景下仔细评估相关性假设,而非假设其始终存在。受此启发,我们提出两种自适应压缩设计,根据测量的相关性强度在不同压缩模式间切换,并评估其相对于传统非自适应方法的性能提升。总之,我们的统一分类法为开发更有效且应用特定的联邦学习压缩技术提供了清晰且原则性的基础。

英文摘要:

The communication bottleneck in federated learning (FL) has spurred extensive research into techniques to reduce the volume of data exchanged between client devices and the central parameter server. In this paper, we systematically classify gradient and model compression schemes into three categories based on the type of correlations they exploit: structural, temporal, and spatial. We examine the sources of such correlations, propose quantitative metrics for measuring their magnitude, and reinterpret existing compression methods through this unified correlation-based framework. Our experimental studies demonstrate that the degrees of structural, temporal, and spatial correlations vary significantly depending on task complexity, model architecture, and algorithmic configurations. These findings suggest that algorithm designers should carefully evaluate correlation assumptions under specific deployment scenarios rather than assuming that they are always present. Motivated by these findings, we propose two adaptive compression designs that actively switch between different compression modes based on the measured correlation strength, and we evaluate their performance gains relative to conventional non-adaptive approaches. In summary, our unified taxonomy provides a clean and principled foundation for developing more effective and application-specific compression techniques for FL systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑