arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.07190math.OC
收藏

分散式个性化联邦学习:所有个性化模式的下界和最优算法

Decentralized Personalized Federated Learning: Lower Bounds and Optimal Algorithm for All Personalization Modes

Abdurakhmon Sadiev, Ekaterina Borodich, Aleksandr Beznosikov, Darina Dvinskikh, Saveliy Chezhegov, Rachael Tappenden, Martin Takáč, Alexander Gasnikov

摘要:

本文考虑了分散的、个性化的联邦学习问题。对于集中式个性化联邦学习,通常会在目标函数中添加一个惩罚,用于测量与局部模型及其平均值的偏差。然而,在去中心化的环境中,这种惩罚在通信成本方面是昂贵的,因此在这里,使用不同的惩罚-一种被构建为尊重底层计算网络的结构的惩罚。我们提出了下界的通信和本地计算成本为这个问题的制定,我们还提出了可证明的分散个性化联邦学习的最佳方法。数值实验证明了我们的方法的实际性能。

英文摘要:

This paper considers the problem of decentralized, personalized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the local model and its average, is often added to the objective function. However, in a decentralized setting this penalty is expensive in terms of communication costs, so here, a different penalty - one that is built to respect the structure of the underlying computational network - is used instead. We present lower bounds on the communication and local computation costs for this problem formulation and we also present provably optimal methods for decentralized personalized federated learning. Numerical experiments are presented to demonstrate the practical performance of our methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑