通信约束下公平且个性化的去中心化学习统一框架
A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints
浏览论文内容
中文总结 AI 辅助
该研究针对通信约束下去中心化学习的公平性等挑战，提出融合个性化、公平性与高效通信的DMFL-SQ算法，在CIFAR-10等数据集上验证其能减少通信并维持性能、提升公平性。
中文摘要 AI 辅助
去中心化学习系统旨在在不依赖中央协调器的情况下，跨多个客户端协同训练模型。去中心化虽提升了可扩展性、隐私性和鲁棒性，但也加剧了三大核心挑战：客户端间的统计异质性、客户端层面性能的公平性以及严格的通信约束。这引出一个自然问题：在有限通信下，去中心化学习能达到何种公平性？我们通过提出通信约束下去中心化学习的统一框架来解决该问题，该框架融合了基于图的个性化、不可知公平性以及压缩事件触发通信。具体而言，我们提出了新算法DMFL-SQ，这是一种去中心化多任务学习算法，它将通信图上的个性化模型训练与不可知混合公平性目标相结合，同时通过稀疏化、量化和事件触发同步减少通信。我们为一般非凸目标建立了收敛保证，表明尽管存在稀疏、量化和事件触发的通信，DMFL-SQ在期望平方Moreau包络平稳性上达到了O(T^{-1/2})的速率。我们进一步为公平感知混合目标推导了PAC-贝叶斯泛化保证。在CIFAR-10和真实异构MUSMET EEG数据集上的实验表明，DMFL-SQ在大幅减少通信的同时，保持了预测性能并提升了客户端间的公平性。我们的理论和实验结果共同表明，在去中心化学习中可同时实现个性化、公平性和通信效率，且能保持主导收敛速率。
英文摘要
Decentralized learning systems aim to collaboratively train models across multiple clients without relying on a central coordinator. While decentralization improves scalability, privacy, and robustness, it also exacerbates three fundamental challenges: statistical heterogeneity across clients, fairness in client-level performance, and stringent communication constraints. This raises a natural question: \emph{how fair can decentralized learning be under limited communication?} We address this question by presenting a unified framework for decentralized learning under communication constraints, bringing together graph-based personalization, agnostic fairness, and compressed event-triggered communication. Specifically, we propose a new algorithm DMFL-SQ, a decentralized multi-task learning algorithm that couples personalized model training over a communication graph with an agnostic mixture fairness objective, while reducing communication through sparsification, quantization, and event-triggered synchronization. We establish convergence guarantees for general non-convex objectives and show that DMFL-SQ achieves an $\mathcal{O}(T^{-1/2})$ rate in expected squared Moreau-envelope stationarity despite sparse, quantized, and event-triggered communication. We further derive PAC-Bayes generalization guarantees for the fairness-aware mixture objective. Experiments on CIFAR-10 and the real heterogeneous MUSMET EEG dataset demonstrate that DMFL-SQ substantially reduces communication while maintaining predictive performance and improving fairness across clients. Together, our theoretical and empirical results show that personalization, fairness, and communication efficiency can be jointly achieved in decentralized learning while preserving the dominant convergence rate.
发表机构
- School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology（瑞典皇家理工学院电气与计算机科学学院）
机构由 AI 辅助整理，请以论文原文为准。
