arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.07289cs.LGcs.DCmath.OC
收藏

面向极小-极大问题的去中心化个性化联邦学习

Decentralized Personalized Federated Learning for Min-Max Problems

Moscow Institute of Physics and Technology(莫斯科物理技术学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Ivannikov Institute for System Programming RAS(俄罗斯科学院伊万尼科夫系统编程研究所);Bosch Center for Artificial Intelligence(博世人工智能中心)

Ekaterina Borodich, Aleksandr Beznosikov, Abdurakhmon Sadiev, Vadim Sushko, Nikolay Savelyev, Martin Takáč, Alexander Gasnikov

AI总结:

针对现有个性化联邦学习理论多聚焦极小化问题的局限,本文提出去中心化混合目标PFL框架及新算法,完成凸凹鞍点问题理论分析,实验验证了方法有效性。

摘要:

个性化联邦学习(Personalized Federated Learning, PFL)已取得显著进展,推动了可保护训练数据隐私的创新机器学习应用的发展。但该领域现有理论研究主要聚焦于极小化问题的分布式优化。本文首次研究涵盖更广泛优化问题类的鞍点问题的PFL,这类问题无法仅通过求解极小化问题解决。本研究采用近期提出的带混合目标函数的PFL设定,该方法将全局模型学习与本地分布式学习器相结合。与多数仅考虑中心化设定的既往研究不同,本文采用更通用的去中心化架构,可设计并分析更实用、更符合联邦特性的设备组网方式。本文提出了求解该问题的新算法,并针对随机与确定性场景下的光滑(强)凸-(强)凹鞍点问题提供了理论分析。针对双线性问题和带对抗噪声的神经网络的数值实验,验证了所提方法的有效性。

英文摘要:

Personalized Federated Learning (PFL) has witnessed remarkable advancements, enabling the development of innovative machine learning applications that preserve the privacy of training data. However, existing theoretical research in this field has primarily focused on distributed optimization for minimization problems. This paper is the first to study PFL for saddle point problems encompassing a broader range of optimization problems, that require more than just solving minimization problems. In this work, we consider a recently proposed PFL setting with the mixing objective function, an approach combining the learning of a global model together with locally distributed learners. Unlike most previous work, which considered only the centralized setting, we work in a more general and decentralized setup that allows us to design and analyze more practical and federated ways to connect devices to the network. We proposed new algorithms to address this problem and provide a theoretical analysis of the smooth (strongly) convex-(strongly) concave saddle point problems in stochastic and deterministic cases. Numerical experiments for bilinear problems and neural networks with adversarial noise demonstrate the effectiveness of the proposed methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑