We consider online model selection with decentralized data over $M$ clients, and study the necessity of collaboration among clients. Previous work proposed various federated algorithms without demonstrating their necessity,while we answer the question from a novel perspective of computational constraints. We prove lower bounds on the regret, and propose a federated algorithm and analyze the upper bound.Our results show (i) collaboration is unnecessary in the absence of computational constraints on clients; (ii) collaboration is necessary if the computational cost on each client is limited to $o(K)$, where $K$ is the number of candidate hypothesis spaces. We clarify the unnecessary nature of collaboration in previous federated algorithms for distributed online multi-kernel learning,and improve the regret bounds at a smaller computational and communication cost. Our algorithm relies on three new techniques including an improved Bernstein's inequality for martingale, a federated online mirror descent framework, and decoupling model selection and prediction, which might be of independent interest.
Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs
GE – Global Research(通用电气全球研究中心);Inria(法国国家信息与自动化研究所);Univ Lyon(里昂大学);UJM-Saint-Etienne(圣艾蒂安让·莫内大学);CNRS(法国国家科学研究中心);Laboratoire Hubert Curien(于贝尔·居里安实验室)
Robust Decentralized Federated Distillation via Multi-Modality Knowledge Collaboration
Faculty of Applied Sciences, Macao Polytechnic University(澳门理工大学应用科学学院);School of Engineering and Technology, Central Queensland University(中央昆士兰大学工程与技术学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaboration
Faculty of Applied Sciences, Macao Polytechnic University(澳门理工大学应用科学学院);School of Engineering and Technology, Central Queensland University(中央昆士兰大学工程与技术学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Decentralized Directed Collaboration for Personalized Federated Learning
Nanjing University of Science and Technology(南京理工大学);Tsinghua University(清华大学);National University of Defense Technology(国防科技大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));JD Explore Academy(京东探索研究院)
King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学)
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution
School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);Shandong Computer Science Center (National Supercomputer Center in Jinan)(山东省计算中心(国家超级计算济南中心));Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));Shandong Fundamental Research Center for Computer Science(山东省计算机科学基础研究中心)
Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness
University of Tokyo(东京大学)
更广泛的覆盖范围和更好的解决方案,以减少5G的延迟,需要它与多路访问边缘计算(MEC)技术相结合。分散式深度学习(Decentralized Deep Learning,简称DAI),如联邦学习(Federated Learning)和群学习(Swarm Learning),作为数百万智能边缘设备的隐私保护数据处理的一种有前途的解决方案,利用了本地客户端网络内多层神经网络的分布式计算,而不会泄露原始的本地训练数据。值得注意的是,在金融和医疗保健等行业,交易和个人医疗记录的敏感数据被谨慎地保存着,ESTA可以促进这些机构之间的合作,以提高训练模型的性能,同时保护参与客户的数据隐私。在这篇调查论文中,我们展示了分布式学习的技术基础,这些技术基础通过分散式学习使社会各界受益。此外,我们还从通信效率和可信度的新角度概述了物联网的挑战和最相关的解决方案,从而全面概述了该领域的当前最新技术水平。
Task-Agnostic Federation over Decentralized Data: Research Landscape and Visions
Department of Computer Science, College of Information Science and Technology, Jinan University(计算机科学系,信息科学与技术学院,暨南大学);Department of Computer Science, University of Warwick(计算机科学系,沃里克大学);School of Electronic Engineering and Computer Science, Queen Mary University of London(电子工程与计算机科学学院,伦敦女王玛丽大学);Department of Electronic Business and the Institute of Digital Business & Intelligent Logistics, South China University of Technology(电子商务系及数字商务与智能物流研究所,华南理工大学);College of Computer Science and Software Engineering and the Shenzhen Institute of Computing Sciences, Shenzhen University(计算机科学与软件工程学院及深圳计算科学研究所,深圳大学);Department of Computer Science, State University of New York(计算机科学系,纽约州立大学)
Delft University of Technology(代尔夫特理工大学);ENSTA Paris(巴黎国立高等先进技术学校);Institut Polytechnique de Paris(巴黎理工学院);Shell Global Solutions International B.V.(壳牌全球解决方案国际有限公司)
Adaptive collaboration for online personalized distributed learning with heterogeneous clients
Argo Team, Inria Paris - Département d’informatique de l’ENS, PSL Research University(Argo团队,Inria巴黎,信息学院,PSL研究大学);Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189, CRIStAL(里尔大学,Inria,国家科学研究中心,里尔中央理工大学,UMR 9189,CRIStAL)
Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms
1 Centre Inria d'Université Côte d'Azur, INRIA, Sophia Antipolis, France.;2 Laboratoire GAATI, Universit\' e de la Polyn\' e sie fran c aise, Fa`a`\= a , French Polynesia.;3 School of Electrical;Electronic Engineering, University of Sheffield, Sheffield, United Kingdom.;4 ECE Dept. Princeton University, Princeton, 08544 NJ, USA.
DeceFL: A Principled Decentralized Federated Learning Framework
Huazhong University of Science and Technology(华中科技大学);University of Waterloo(滑铁卢大学);Northeastern University(东北大学);KTH Royal Institute of Technology(瑞典皇家理工学院)
Confederated Learning: Federated Learning with Decentralized Edge Servers
University of Electronic Science and Technology of China(电子科技大学);Stevens Institute of Technology(史蒂文斯理工学院);Sun Yat-Sen University(中山大学);Peng Cheng Laboratory(鹏城实验室)
Efficient Collaborations through Weight-Driven Coalition Dynamics in Federated Learning Systems
ENSIAS, Mohammed V University in Rabat(拉巴特穆罕默德五世大学ENSIAS学院);Higher School of Technology, Hassan II University in Casablanca(卡萨布兰卡哈桑二世大学高等技术学院);School of Science and Engineering, Al Akhawayn University in Ifrane(伊夫兰阿卡韦恩大学科学与工程学院)