联邦学习中的自适应行列式点过程客户端调度
Adaptive Determinantal Client Scheduling in Federated Learning
浏览论文内容
中文总结 AI 辅助
针对联邦学习中客户端调度忽视多样性的问题，提出基于行列式点过程的自适应调度算法ADCS，实现质量-多样性权衡，并给出收敛性分析，实验证明其优于现有算法。
中文摘要 AI 辅助
在联邦学习中，由于数据和系统的异构性，调度客户端进行模型训练至关重要。以往的大多数工作侧重于调度客户端的质量，以实现更快的收敛、更短的墙钟收敛时间或更好的平均模型性能。它们很少考虑客户端的多样性，而多样性对于对抗异构性并改善最差客户端的性能非常重要。在这项工作中，我们主张使用行列式点过程（DPPs）来建模并增强客户端调度的多样性。我们首先利用梯度信息和质量分数设计DPPs的核矩阵，这固有地实现了灵活的质量-多样性权衡。通过对DPPs应用快速最大后验（MAP）推断，我们提出了联邦学习中的自适应行列式点过程客户端调度（ADCS）。我们进一步量化了ADCS的梯度近似误差，并为联邦学习中具有非凸损失函数的一般有偏客户端选择开发了收敛性分析。我们进行了对比数值实验，表明ADCS优于最先进的客户端调度算法，包括基于质量和基于多样性的算法。
英文摘要
Scheduling clients for model training is critical in federated learning due to both data and system heterogeneity. Most previous works focus on the quality of the scheduled clients to achieve faster convergence, shorter wall-clock convergence time, or better average model performance. They rarely consider the diversity of clients, which is important to counter heterogeneity and improve performance for the worst-off clients. In this work, we advocate the use of determinantal point processes (DPPs) to model and enhance the diversity in client scheduling. We first design the kernel matrices of DPPs using gradient information and quality scores, which inherently enables a flexible quality-diversity trade-off. Applying fast MAP inference over DPPs, we propose Adaptive Determinantal Client Scheduling (ADCS) in FL. We further quantify the gradient approximation error of ADCS and develop convergence analysis for general biased client selection in FL with non-convex loss functions. We conduct comparative numerical experiments showing that ADCS outperforms state-of-the-art client scheduling algorithms, including both quality-based and diversity-based ones.
发表机构
- University of Toronto（多伦多大学）
- Ericsson（爱立信）
机构由 AI 辅助整理，请以论文原文为准。
