The ultra-low latency requirements of 5G/6G applications and privacy constraints call for distributed machine learning systems to be deployed at the edge. With its simple yet effective approach, federated learning (FL) is a natural solution for massive user-owned devices in edge computing with distributed and private training data. FL methods based on FedAvg typically follow a naive star topology, ignoring the heterogeneity and hierarchy of the volatile edge computing architectures and topologies in reality. Several other network topologies exist and can address the limitations and bottlenecks of the star topology. This motivates us to survey network topology-related FL solutions. In this paper, we conduct a comprehensive survey of the existing FL works focusing on network topologies. After a brief overview of FL and edge computing networks, we discuss various edge network topologies as well as their advantages and disadvantages. Lastly, we discuss the remaining challenges and future works for applying FL to topology-specific edge networks.
Federated learning usually employs a client-server architecture where an orchestrator iteratively aggregates model updates from remote clients and pushes them back a refined model. This approach may be inefficient in cross-silo settings, as close-by data silos with high-speed access links may exchange information faster than with the orchestrator, and the orchestrator may become a communication bottleneck. In this paper we define the problem of topology design for cross-silo federated learning using the theory of max-plus linear systems to compute the system throughput---number of communicatio…
The convergence speed of machine learning models trained with Federated Learning is significantly affected by heterogeneous data partitions, even more so in a fully decentralized setting without a central server. In this paper, we show that the impact of label distribution skew, an important type of data heterogeneity, can be significantly reduced by carefully designing the underlying communication topology. We present D-Cliques, a novel topology that reduces gradient bias by grouping nodes in sparsely interconnected cliques such that the label distribution in a clique is representative of the…
PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmark
Shanghai Jiao Tong University（上海交通大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Queen’s University Belfast（贝尔法斯特女王大学）；Stevens Institute of Technology（史蒂文斯理工学院）
A Comprehensive Survey On Client Selections in Federated Learning
College of Computing and Information Technology, University of Doha for Science and Technology（多哈科技大学计算与信息技术学院）；Electrical Engineering, Qatar University（卡塔尔大学电气工程系）
Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review
Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, India（人工智能系，印度理工学院海得拉巴学院）；Department of Computing Technologies, Swinburne University of Technology, Hawthorn, australia（计算技术系，斯威本技术大学 Hawthorn 分校）；Department of Physics, Indian Institute of Technology Delhi, India（物理系，印度理工学院德里学院）；Department of Energy Science and Engineering, Indian Institute of Technology Delhi, India（能源科学与工程系，印度理工学院德里学院）；Department of Computer Science and Engineering, Indian Institute of Technology Hyderabad, India（计算机科学与工程系，印度理工学院海得拉巴学院）
Purdue University（普渡大学）；Arizona State University（亚利桑那州立大学）
Federated learning has emerged as a popular technique for distributing model training across the network edge. Its learning architecture is conventionally a star topology between the devices and a central server. In this paper, we propose two timescale hybrid federated learning (TT-HF), which migrates to a more distributed topology via device-to-device (D2D) communications. In TT-HF, local model training occurs at devices via successive gradient iterations, and the synchronization process occurs at two timescales: (i) macro-scale, where global aggregations are carried out via device-server int…
From Models to Network Topologies: A Topology Inference Attack in Decentralized Federated Learning
Communication Systems Group, Department of Informatics, University of Zurich, CH--8050 Zürich, Switzerland（苏黎世大学信息学院通信系统组）；Department of Information and Communications Engineering, University of Murcia, 30100--Murcia, Spain（穆尔西亚大学信息与通信工程系）
Northeastern University（东北大学）；HKUST（香港科技大学）；Hong Kong Baptist University（香港浸会大学）
Federated learning utilizes various resources provided by participants to collaboratively train a global model, which potentially address the data privacy issue of machine learning. In such promising paradigm, the performance will be deteriorated without sufficient training data and other resources in the learning process. Thus, it is quite crucial to inspire more participants to contribute their valuable resources with some payments for federated learning. In this paper, we present a comprehensive survey of incentive schemes for federate learning. Specifically, we identify the incentive probl…
By leveraging deep learning based technologies, the data-driven based approaches have reached great success with the rapid increase of data generated of Industrial Indernet of Things(IIot). However, security and privacy concerns are obstacles for data providers in many sensitive data-driven industrial scenarios, such as healthcare and auto-driving. Many Federated Learning(FL) approaches have been proposed with DNNs for IIoT applications, these works still suffer from low usability of data due to data incompleteness, low quality, insufficient quantity, sensitivity, etc. Therefore, we propose a …
The University of Hong Kong（香港大学）；Southern University of Science and Technology（南方科技大学）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Peng Cheng Laboratory（鹏城实验室）
Federated edge learning (FEEL) is envisioned as a promising paradigm to achieve privacy-preserving distributed learning. However, it consumes excessive learning time due to the existence of straggler devices. In this paper, a novel topology-optimized federated edge learning (TOFEL) scheme is proposed to tackle the heterogeneity issue in federated learning and to improve the communication-and-computation efficiency. Specifically, a problem of jointly optimizing the aggregation topology and computing speed is formulated to minimize the weighted summation of energy consumption and latency. To sol…
A recent emphasis of distributed learning research has been on federated learning (FL), in which model training is conducted by the data-collecting devices. Existing research on FL has mostly focused on a star topology learning architecture with synchronized (time-triggered) model training rounds, where the local models of the devices are periodically aggregated by a centralized coordinating node. However, in many settings, such a coordinating node may not exist, motivating efforts to fully decentralize FL. In this work, we propose a novel methodology for distributed model aggregations via asy…
Topology Learning for Heterogeneous Decentralized Federated Learning over Unreliable D2D Networks
School of Computer Science and Technology, Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；University of Electronic Science and Technology of China（电子科技大学）
OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework
Meituan（美团）；Bytedance Inc.（字节跳动公司）；Tsinghua University（清华大学）；University of Macau（澳门大学）；State Key Lab of Computer Science, ISCAS（中国科学院软件研究所计算机科学国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Personalized Federated Learning (pFL), which utilizes and deploys distinct local models, has gained increasing attention in recent years due to its success in handling the statistical heterogeneity of FL clients. However, standardized evaluation and systematical analysis of diverse pFL methods remain a challenge. Firstly, the highly varied datasets, FL simulation settings and pFL implementations prevent easy and fair comparisons of pFL methods. Secondly, the current pFL literature diverges in the adopted evaluation and ablation protocols. Finally, the effectiveness and robustness of pFL method…
Towards Heterogeneity-Aware and Energy-Efficient Topology Optimization for Decentralized Federated Learning in Edge Environment
School of Engineering, Swinburne University of Technology（工程学院，斯威本科技大学）；School of Computer Science and Technology, Tongji University（计算机科学与技术学院，同济大学）；State Key Lab for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；School of Computer Science and Artificial Intelligence, Wuhan University of Technology（计算机科学与人工智能学院，武汉科技大学）；School of Cyber Science and Engineering, Wuhan University（网络科学与工程学院，武汉大学）；CSIRO Data61
Federated Learning Architectures: A Performance Evaluation with Crop Yield Prediction Application
Department of Computer Science Mahishadal Raj College, Mahishadal, West Bengal, 721628, India；Distributed Systems (CLOUDS) Laboratory, \ of Computing；Information Systems, The University of Melbourne, Victoria, 3052, Australia
Hong Kong University of Science and Technology（香港科技大学）；Webank（微众银行）；Beihang University（北京航空航天大学）
Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework…
Department of Computer, Control and Management Engineering, Sapienza University of Rome（罗马萨皮恩扎大学计算机、控制与管理工程系）；Department of Computer Science, Sapienza University of Rome（罗马萨皮恩扎大学计算机科学系）；Università della Svizzera Italiana（意大利瑞士大学）
Quantum Federated Learning: A Comprehensive Survey
Department of Electrical and Computer Engineering, The University of Alabama in Huntsville, USA（电气与计算机工程系，美国阿拉巴马大学亨茨维尔分校）；Faculty of Engineering and Applied Science, Memorial University, Canada（工程与应用科学学院，加拿大纪念大学）；College of Computer Science and Engineering, Nanyang Technological University, Singapore（计算机科学与工程学院，新加坡南洋理工大学）
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Chungnam National University（忠南大学）
Federated learning has emerged as a new paradigm of collaborative machine learning; however, many prior studies have used global aggregation along a star topology without much consideration of the communication scalability or the diurnal property relied on clients' local time variety. In contrast, ring architecture can resolve the scalability issue and even satisfy the diurnal property by iterating nodes without an aggregation. Nevertheless, such ring-based algorithms can inherently suffer from the high-variance problem. To this end, we propose a novel algorithm called TornadoAggregate that im…
Federated Learning in Mobile Networks: A Comprehensive Case Study on Traffic Forecasting
Democritus University of Thrace（德谟克利特色雷斯大学）；Imperial College London（帝国理工学院）；Centre Tecnològic de Telecomunicacions de Catalunya (CTTC/CERCA)（加泰罗尼亚电信技术中心（CTTC/CERCA））；Nokia Bell Labs（诺基亚贝尔实验室）；IMT Atlantique（大西洋电信学校）；Inria（法国国家信息与自动化研究所）；LS2N（南特计算机科学与实验室（注：此处LS2N为实验室，按要求保留独立机构名））
Topology-Aware Differential Privacy in Hierarchical Federated Learning
Quantum Cloud Computing and Distributed Systems (qCLOUDS) Lab, School of Computing and Information Systems, The University of Melbourne（量子云计算与分布式系统实验室，计算与信息系统学院，墨尔本大学）
DFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learning
Shandong University, School of Mechanical, Electrical and Information Engineering（山东大学，机械电子信息工程学院）；Beijing Institute of Technology, School of Computer Science and Technology（北京理工大学，计算机科学与技术学院）；Sun Yat-sen University, School of Computer Science and Engineering（中山大学，计算机科学与工程学院）
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；National University of Singapore（新加坡国立大学）
Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the parameter updates of models during training. We design inference algorithms for both centralized and federated learning, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference at…
