Over the past few years, the federated learning ($\texttt{FL}$) community has witnessed a proliferation of new $\texttt{FL}$ algorithms. However, our understating of the theory of $\texttt{FL}$ is still fragmented, and a thorough, formal comparison of these algorithms remains elusive. Motivated by this gap, we show that many of the existing $\texttt{FL}$ algorithms can be understood from an operator splitting point of view. This unification allows us to compare different algorithms with ease, to refine previous convergence results and to uncover new algorithmic variants. In particular, our analysis reveals the vital role played by the step size in $\texttt{FL}$ algorithms. The unification also leads to a streamlined and economic way to accelerate $\texttt{FL}$ algorithms, without incurring any communication overhead. We perform numerical experiments on both convex and nonconvex models to validate our findings.
Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation
School of Optoelectronic & Communication Engineering, Xiamen University of Technology(厦门理工学院光电信息与通信工程学院);National Key Laboratory of Wireless Communications, University of Electronic Science and Technology of China(电子科技大学信息与通信国家重点实验室);Division of Information Science and Engineering, KTH Royal Institute of Technology(皇家理工学院信息科学与工程系);School of Cyber Science and Engineering, Huazhong University of Science and Technology(华中科技大学网络安全科学与工程学院);School of Engineering, Huaqiao University(华侨大学工程学院)
ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada(电气与计算机工程系,不列颠哥伦比亚大学);Department of Computer Science and Engineering, Southern University of Science and Technology, China(计算机科学与工程系,南方科技大学)
FedCVU: Federated Learning for Cross-View Video Understanding
Software College, Northeastern University, Shenyang, China(东北大学软件学院);University of Science and Technology of China, Hefei, China(中国科学技术大学);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学);Zhejiang University, Hangzhou, China(浙江大学)
Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach
Carnegie Mellon University(卡内基梅隆大学);Massachusetts Institute of Technology(麻省理工学院);Columbia University(哥伦比亚大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Petuum Inc.(Petuum公司)
联邦学习的规范公式将其视为分布式优化问题,其中模型参数针对全局损失函数进行优化,该全局损失函数在客户端损失函数之间进行分解。最近的替代公式将联邦学习视为分布式推理问题,其中目标是从分区的客户端数据推断全局后验(Al-Shedivat et al.,2021年)。本文扩展了推理视图,并描述了变分推理公式的联邦学习的目标是找到一个全局变分后验,以及近似的真实后验。这自然激发了联邦学习(FedEP)的期望传播方法,其中通过中央服务器和客户端之间的概率消息传递迭代地细化全局后验的近似值。我们进行了广泛的实证研究,在各种算法的考虑,并描述了实际的战略,扩大期望传播到现代联邦设置。我们将FedEP应用于标准的联邦学习基准测试,发现它在收敛速度和准确性方面都优于强基线。
Principled Federated Random Forests for Heterogeneous Data
Inria(法国国家信息与自动化研究所);PreMeDICaL(预医学实验室);Inserm(法国国家医学研究院);University of Montpellier(蒙彼利埃大学);Sorbonne Université(索邦大学);Université Paris Cité(巴黎城市大学);CNRS(国家科学研究中心);LPSM(巴黎高等师范学院)
No Free Lunch Theorem for Security and Utility in Federated Learning
Hong Kong University of Science and Technology(香港科技大学);Webank(微众银行)
在联邦学习场景中,多方从各自的数据中联合学习模型,对于选择合适的算法存在两个相互冲突的目标。一方面,在\textit{semi-honest}合作伙伴在场的情况下,私人和敏感的训练数据必须尽可能地保持安全,而另一方面,为了学习效用,必须在不同方之间交换一定量的信息。这样的挑战需要隐私保护的联邦学习解决方案,它最大限度地提高了学习模型的效用,并保持了参与方的私人数据的可证明的隐私保证。本文阐述了一个一般框架,a)从统一的信息论角度阐述了隐私损失和效用损失之间的权衡,b)描绘了使用不同保护机制(包括随机化、稀疏性和同态加密)时隐私-效用权衡的定量界限。结果表明,在一般情况下,\textit{there is no free lunch for the privacy-utility trade-off}和一个交易的隐私保护与一定程度的降级效用。本文的定量分析对设计实用的联邦学习算法具有指导意义。
FedMood: Federated Learning on Mobile Health Data for Mood Detection
School of Cyber Science and Technology, Beihang University(北京航空航天大学网络空间科学与技术学院);Beijing Advanced Innovation Center for Big Data and Brain Computing(北京大数据与脑计算高精尖创新中心);University of Illinois at Chicago(伊利诺伊大学芝加哥分校);Fordham University(福特汉姆大学);Lehigh University(理海大学)
MSfusion: A Dynamic Model Splitting Approach for Resource-Constrained Machines to Collaboratively Train Larger Models
Internet of Things Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)物联网部);School of Cyber Science and Engineering, Southeast University(东南大学网络空间科学与工程学院)
FedABC: Attention-Based Client Selection for Federated Learning with Long-Term View
Advanced Wireless Technology Laboratory, Munich Research Center, Huawei Technologies Duesseldorf GmbH(慕尼黑研究中心先进无线技术实验室);TUM School of Computation, Information and Technology, Technical University of Munich(慕尼黑技术大学计算、信息与技术学院);Wireless Technology Lab, 2012 Laboratories, Huawei Technologies Co., Ltd(华为技术有限公司2012实验室)
MammoFL: Mammographic Breast Density Estimation using Federated Learning
University of Pennsylvania(宾夕法尼亚大学);Massachusetts Institute of Technology(麻省理工学院);GSK(葛兰素史克);Indiana University School of Medicine(印第安纳大学医学院);Mayo Clinic(梅奥诊所)