Decentralized and federated learning algorithms face data heterogeneity as one of the biggest challenges, especially when users want to learn a specific task. Even when personalized headers are used concatenated to a shared network (PF-MTL), aggregating all the networks with a decentralized algorithm can result in performance degradation as a result of heterogeneity in the data. Our algorithm uses exchanged gradients to calculate the correlations among tasks automatically, and dynamically adjusts the communication graph to connect mutually beneficial tasks and isolate those that may negatively impact each other. This algorithm improves the learning performance and leads to faster convergence compared to the case where all clients are connected to each other regardless of their correlations. We conduct experiments on a synthetic Gaussian dataset and a large-scale celebrity attributes (CelebA) dataset. The experiment with the synthetic data illustrates that our proposed method is capable of detecting tasks that are positively and negatively correlated. Moreover, the results of the experiments with CelebA demonstrate that the proposed method may produce significantly faster training results than fully-connected networks.
DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training
University of Science and Technology of China(中国科学技术大学);JD Explore Academy(京东探索研究院);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Institute of Artificial Intelligence(人工智能研究院)
King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学)
Towards Personalized Differentially Private Learning for Decentralized Local Graphs
School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(北京邮电大学网络与交换技术国家重点实验室);School of Cyber Science and Technology, Shandong University(山东大学网络空间安全学院);School of Cyberspace Security, Beijing University of Posts and Telecommunications(北京邮电大学网络空间安全学院);Department of Computer Science, University of Illinois at Chicago(伊利诺伊大学芝加哥分校计算机科学系)
Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaboration
Faculty of Applied Sciences, Macao Polytechnic University(澳门理工大学应用科学学院);School of Engineering and Technology, Central Queensland University(中央昆士兰大学工程与技术学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Decentralized Directed Collaboration for Personalized Federated Learning
Nanjing University of Science and Technology(南京理工大学);Tsinghua University(清华大学);National University of Defense Technology(国防科技大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));JD Explore Academy(京东探索研究院)
Resource-Constrained Decentralized Federated Learning via Personalized Event-Triggering
Elmore Family School of Electrical and Computer Engineering, Purdue University(电气与计算机工程学院,普渡大学);Department of Electrical Engineering, University at Buffalo–SUNY(电气工程系,布法罗大学–SUNY)
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training
Tsinghua University(清华大学);Nanjing University of Science and Technology(南京理工大学);The University of Sydney(悉尼大学);Virginia Tech(弗吉尼亚理工大学);JD Explore Academy(京东探索研究院)
UA-PDFL: A Personalized Approach for Decentralized Federated Learning
School of Artificial Intelligence and Computer Science, Jiangnan University(江南大学人工智能与计算机学院);Jiangsu Key University Laboratory of Software and Media Technology under Human-Computer Cooperation (Jiangnan University)(江南大学江苏省人机协同软件与媒体技术重点实验室)
Decentralized Personalized Federated Learning for Min-Max Problems
Moscow Institute of Physics and Technology(莫斯科物理技术学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Ivannikov Institute for System Programming RAS(俄罗斯科学院伊万尼科夫系统编程研究所);Bosch Center for Artificial Intelligence(博世人工智能中心)
Decentralized Federated Learning: Balancing Communication and Computing Costs
分散式随机梯度下降(SGD)是分散式联邦学习(DFL)的驱动引擎。分散式SGD的性能受到节点间通信和本地更新的共同影响。在本文中,我们提出了一个通用的DFL框架,它实现了多个本地更新和多个节点间的通信周期,以达到通信效率和模型共识之间的平衡。它可以提供一个通用的去中心化SGD分析框架。我们为所提出的DFL算法建立了强收敛保证,而无需假设凸目标。DFL的收敛速度可以被优化,以实现在有限的资源下的通信和计算成本的平衡。为了提高DFL的通信效率,将压缩通信作为一种新的方案引入到DFL中,命名为DFL with compressed communication(C-DFL)。建议的C-DFL表现出线性收敛的强凸目标。基于MNIST和CIFAR-10数据集的实验结果表明,DFL优于传统的分散SGD方法,并表明C-DFL进一步提高了通信效率。
Semantic Communication in Dynamic Channel Scenarios: Collaborative Optimization of Dual-Pipeline Joint Source-Channel Coding and Personalized Federated Learning
Beijing Institute of Technology(北京理工大学);Institute of Computer Application Technology Norinco Groups(中国兵器工业集团计算机应用技术研究所)
GFL: A Decentralized Federated Learning Framework Based On Blockchain
Polytechnic Institute, Zhejiang University(浙江大学理工学院);School of Software Technology, Zhejiang University(浙江大学软件学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Public Affairs, Zhejiang University(浙江大学公共管理学院)
LoDAdaC: a unified local training-based decentralized framework with adaptive gradients and compressed communication
Rensselaer Polytechnic Institute(伦斯勒理工学院);IBM T. J. Watson Research Center(IBM T. J. Watson 研究中心);MIT-IBM Watson AI Lab, IBM Research(MIT-IBM Watson AI 实验室,IBM 研究院)
Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs
GE – Global Research(通用电气全球研究中心);Inria(法国国家信息与自动化研究所);Univ Lyon(里昂大学);UJM-Saint-Etienne(圣艾蒂安让·莫内大学);CNRS(法国国家科学研究中心);Laboratoire Hubert Curien(于贝尔·居里安实验室)
Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning
Tsinghua University(清华大学);Peking University(北京大学);University of Bristol(布里斯托大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));The University of Melbourne(墨尔本大学);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学)
PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillation
Doctoral School of Exact and Natural Sciences, Jagiellonian University(杰哥利昂大学精确与自然科学博士学院);Faculty of Mathematics and Computer Science, Jagiellonian University(杰哥利昂大学数学与计算机科学学院)