arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.05090cs.LGcs.DCmath.OC
收藏

表现性联邦学习:解决模型依赖和异构分布偏移的方法

Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shifts

University of Michigan(密歇根大学);The Ohio State University(俄亥俄州立大学);University of California, Santa Cruz(加州大学圣克鲁兹分校);ByteDance AI Lab(字节跳动人工智能实验室)

Kun Jin, Tongxin Yin, Zhongzhu Chen, Zeyu Sun, Xueru Zhang, Yang Liu, Mingyan Liu

AI总结:

针对联邦学习中数据分布受模型影响的问题,提出表现性联邦学习框架,证明表现性FedAvg算法以O(1/T)收敛到稳定解,并分析异构性影响。

摘要:

我们考虑一个由多个客户端和一个服务器组成的联邦学习(FL)系统,其中客户端旨在从其分布式数据中协作学习一个共同的决策模型。与假设客户端数据是静态的传统联邦学习框架不同,我们考虑客户端的数据分布可能被部署的决策模型重塑的场景。在这项工作中,我们利用表现性预测中的分布偏移映射思想来形式化这种模型依赖的数据分布偏移,并提出一个表现性联邦学习框架。我们首先引入存在唯一表现性稳定解的必要和充分条件,并刻画其与表现性最优解的距离。然后我们提出表现性FedAvg算法,并证明在完全参与和部分参与方案下,它都以O(1/T)的速率收敛到表现性稳定解。特别地,我们使用新颖的证明技术,并展示客户端的异构性如何影响收敛。数值结果验证了我们的分析,并为现实世界应用提供了有价值的见解。

英文摘要:

We consider a federated learning (FL) system consisting of multiple clients and a server, where the clients aim to collaboratively learn a common decision model from their distributed data. Unlike the conventional FL framework that assumes the client's data is static, we consider scenarios where the clients' data distributions may be reshaped by the deployed decision model. In this work, we leverage the idea of distribution shift mappings in performative prediction to formalize this model-dependent data distribution shift and propose a performative federated learning framework. We first introduce necessary and sufficient conditions for the existence of a unique performative stable solution and characterize its distance to the performative optimal solution. Then we propose the performative FedAvg algorithm and show that it converges to the performative stable solution at a rate of O(1/T) under both full and partial participation schemes. In particular, we use novel proof techniques and show how the clients' heterogeneity influences the convergence. Numerical results validate our analysis and provide valuable insights into real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑