Federated learning (FL) in a bandwidth-limited network with energy-limited user equipments (UEs) is under-explored. In this paper, to jointly save energy consumed by the battery-limited UEs and accelerate the convergence of the global model in FL for the bandwidth-limited network, we propose the sliding differential evolution-based scheduling (SDES) policy. To this end, we first formulate an optimization that aims to minimize a weighted sum of energy consumption and model training convergence. Then, we apply the SDES with parallel differential evolution (DE) operations in several small-scale windows, to address the above proposed problem effectively. Compared with existing scheduling policies, the proposed SDES performs well in reducing energy consumption and the model convergence with lower computational complexity.
Hitachi (China) Research & Development Cooperation（日立（中国）研发有限公司）
In federated learning (FL), devices contribute to the global training by uploading their local model updates via wireless channels. Due to limited computation and communication resources, device scheduling is crucial to the convergence rate of FL. In this paper, we propose a joint device scheduling and resource allocation policy to maximize the model accuracy within a given total training time budget for latency constrained wireless FL. A lower bound on the reciprocal of the training performance loss, in terms of the number of training rounds and the number of scheduled devices per round, is d…
This paper develops a class of low-complexity device scheduling algorithms for over-the-air federated learning via the method of matching pursuit. The proposed scheme tracks closely the close-to-optimal performance achieved by difference-of-convex programming, and outperforms significantly the well-known benchmark algorithms based on convex relaxation. Compared to the state-of-the-art, the proposed scheme poses a drastically lower computational load on the system: For $K$ devices and $N$ antennas at the parameter server, the benchmark complexity scales with $\left(N^2+K\right)^3 + N^6$ while t…
Owing to the increasing need for massive data analysis and model training at the network edge, as well as the rising concerns about the data privacy, a new distributed training framework called federated learning (FL) has emerged. In each iteration of FL (called round), the edge devices update local models based on their own data and contribute to the global training by uploading the model updates via wireless channels. Due to the limited spectrum resources, only a portion of the devices can be scheduled in each round. While most of the existing work on scheduling focuses on the convergence of…
Federated Learning (FL) is a newly emerged decentralized machine learning (ML) framework that combines on-device local training with server-based model synchronization to train a centralized ML model over distributed nodes. In this paper, we propose an asynchronous FL framework with periodic aggregation to eliminate the straggler issue in FL systems. For the proposed model, we investigate several device scheduling and update aggregation policies and compare their performances when the devices have heterogeneous computation capabilities and training data distributions. From the simulation resul…
In this paper, we study asynchronous federated learning (FL) in a wireless distributed learning network (WDLN). To allow each edge device to use its local data more efficiently via asynchronous FL, transmission scheduling in the WDLN for asynchronous FL should be carefully determined considering system uncertainties, such as time-varying channel and stochastic data arrivals, and the scarce radio resources in the WDLN. To address this, we propose a metric, called an effectivity score, which represents the amount of learning from asynchronous FL. We then formulate an Asynchronous Learning-aware …
Age Aware Scheduling for Differentially-Private Federated Learning
Institute of Communications Engineering, National Yang Ming Chiao Tung University（国立阳明交通大学通信工程研究所）；Simula UiB（西挪威应用科技大学西蒙研究所）；Department of Communication Engineering, National Taipei University（国立台北大学通信工程学系）
Update Estimation and Scheduling for Over-the-Air Federated Learning with Energy Harvesting Devices
Dept. of Electrical and Electronics Engineering, Bilkent University（电子与电子工程系，比尔肯特大学）；Dept. of Electrical and Electronic Engineering, Imperial College London（电子与电子工程系，帝国理工学院伦敦分校）
Princeton University（普林斯顿大学）；Imperial College London（帝国理工学院）
We study federated learning (FL) at the wireless edge, where power-limited devices with local datasets collaboratively train a joint model with the help of a remote parameter server (PS). We assume that the devices are connected to the PS through a bandwidth-limited shared wireless channel. At each iteration of FL, a subset of the devices are scheduled to transmit their local model updates to the PS over orthogonal channel resources, while each participating device must compress its model update to accommodate to its link capacity. We design novel scheduling and resource allocation policies th…
Feature-Based Semantics-Aware Scheduling for Energy-Harvesting Federated Learning
Department of Computer and Information Science（计算机与信息科学系）；Linköping University（林奈技术大学）；Department of Information Engineering（信息工程系）；University of Brescia（布雷西亚大学）；ZJU-UIUC Institute（浙大-伊利诺伊大学联合研究所）；Zhejiang University（浙江大学）
University of Oulu（奥卢大学）；Centre for Wireless Communications（无线通信中心）
The performance of federated learning (FL) over wireless networks depend on the reliability of the client-server connectivity and clients' local computation capabilities. In this article we investigate the problem of client scheduling and resource block (RB) allocation to enhance the performance of model training using FL, over a pre-defined training duration under imperfect channel state information (CSI) and limited local computing resources. First, we analytically derive the gap between the training losses of FL with clients scheduling and a centralized training method for a given training …
Utah State University（犹他州立大学）；University of Wisconsin-Whitewater（威斯康星大学怀特沃特分校）
A new machine learning (ML) technique termed as federated learning (FL) aims to preserve data at the edge devices and to only exchange ML model parameters in the learning process. FL not only reduces the communication needs but also helps to protect the local privacy. Although FL has these advantages, it can still experience large communication latency when there are massive edge devices connected to the central parameter server (PS) and/or millions of model parameters involved in the learning process. Over-the-air computation (AirComp) is capable of computing while transmitting data by allowi…
Green Federated Learning via Carbon-Aware Client and Time Slot Scheduling
Inria Centre at Université Côte d’Azur（法国国家信息与自动化技术研究院（Inria）亚维农分校）；Accenture（埃森哲）；Department of Electrical Engineering (ISY), Linköping University（利厄普大学电气工程系）
Baidu Inc.（百度公司）；Soochow University（苏州大学）；Auburn University（奥本大学）；North Carolina State University（北卡罗来纳州立大学）
Recent years have witnessed a large amount of decentralized data in various (edge) devices of end-users, while the decentralized data aggregation remains complicated for machine learning jobs because of regulations and laws. As a practical approach to handling decentralized data, Federated Learning (FL) enables collaborative global machine learning model training without sharing sensitive raw data. The servers schedule devices to jobs within the training process of FL. In contrast, device scheduling with multiple jobs in FL remains a critical and open problem. In this paper, we propose a novel…
Fairness-Aware Job Scheduling for Multi-Job Federated Learning
Nanyang Technological University（南洋理工大学）；DSO National Laboratories（DSO国家实验室）；Alibaba Group（阿里巴巴集团）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-新加坡南洋理工大学联合研究院）
Utah State University（犹他州立大学）；University of Wisconsin-Whitewater（威斯康星大学怀特沃特分校）
Federated learning (FL) is a highly pursued machine learning technique that can train a model centrally while keeping data distributed. Distributed computation makes FL attractive for bandwidth limited applications especially in wireless communications. There can be a large number of distributed edge devices connected to a central parameter server (PS) and iteratively download/upload data from/to the PS. Due to the limited bandwidth, only a subset of connected devices can be scheduled in each round. There are usually millions of parameters in the state-of-art machine learning models such as de…
Given the abundance and ease of access of personal data today, individual privacy has become of paramount importance, particularly in the healthcare domain. In this work, we aim to utilise patient data extracted from multiple hospital data centres to train a machine learning model without sacrificing patient privacy. We develop a scheduling algorithm in conjunction with a student-teacher algorithm that is deployed in a federated manner. This allows a central model to learn from batches of data at each federal node. The teacher acts between data centres to update the main task (student) algorit…
Non-Coherent Over-the-Air Federated Learning: Protocol, Convergence, and Device Scheduling
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Arizona State University（亚利桑那州立大学）；Northeastern University London（伦敦东北大学）；Shanghai HKU Education Center（上海港大教育中心）
Missouri University of Science and Technology（密苏里科技大学）
The advances in satellite technology developments have recently seen a large number of small satellites being launched into space on Low Earth orbit (LEO) to collect massive data such as Earth observational imagery. The traditional way which downloads such data to a ground station (GS) to train a machine learning (ML) model is not desirable due to the bandwidth limitation and intermittent connectivity between LEO satellites and the GS. Satellite edge computing (SEC), on the other hand, allows each satellite to train an ML model onboard and uploads only the model to the GS which appears to be a…
Dynamic Scheduling for Vehicle-to-Vehicle Communications Enhanced Federated Learning
Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University（北京信息科学与技术国家研究中心，电子工程系，清华大学）；School of Electronic and Information Engineering, Beijing Jiaotong University（电子信息工程学院，北京交通大学）
DPBalance: Efficient and Fair Privacy Budget Scheduling for Federated Learning as a Service
UM-SJTU Joint Institute, Shanghai Jiao Tong University（上海交通大学密西根学院）；Cooperative Medianet Innovation Center (CMIC), Shanghai Jiao Tong University（上海交通大学合作媒体创新中心）；School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）
Army Research Laboratory（陆军研究实验室）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；University of Utah（犹他大学）
Federated learning (FL) is a useful tool in distributed machine learning that utilizes users' local datasets in a privacy-preserving manner. When deploying FL in a constrained wireless environment; however, training models in a time-efficient manner can be a challenging task due to intermittent connectivity of devices, heterogeneous connection quality, and non-i.i.d. data. In this paper, we provide a novel convergence analysis of non-convex loss functions using FL on both i.i.d. and non-i.i.d. datasets with arbitrary device selection probabilities for each round. Then, using the derived conver…
Texas A&M University（德克萨斯农工大学）；Texas A&M University at Qatar（德克萨斯农工大学卡塔尔分校）；Qatar Foundation Research, Development and Innovation RDI（卡塔尔基金会研究、开发与创新（RDI））；Tennessee Tech University（田纳西理工大学）
This paper proposes using communication pipelining to enhance the wireless spectrum utilization efficiency and convergence speed of federated learning in mobile edge computing applications. Due to limited wireless sub-channels, a subset of the total clients is scheduled in each iteration of federated learning algorithms. On the other hand, the scheduled clients wait for the slowest client to finish its computation. We propose to first cluster the clients based on the time they need per iteration to compute the local gradients of the federated learning model. Then, we schedule a mixture of clie…
Virginia Tech（弗吉尼亚理工大学）；University of Colorado Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）；Ericsson Research（爱立信研究院）；Princeton University（普林斯顿大学）；University of Oulu（奥卢大学）
Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks ranging from coordinated trajectory planning to cooperative target recognition. However, due to the lack of continuous connections between the UAV swarm and ground base stations (BSs), using centralized ML will be challenging, particularly when dealing with a large volume of data. In this paper, a novel framework is proposed to implement distributed federated learning (FL) algorithms within a UAV swarm that consists of a leading UAV and several following UAVs. Each following UAV trains a l…
Enhancing Convergence, Privacy and Fairness for Wireless Personalized Federated Learning: Quantization-Assisted Min-Max Fair Scheduling
School of Information and Communication Engineering, Beijing University of Posts and Telecommunications（信息与通信工程学院，北京邮电大学）；School of Computing, Macquarie University（计算学院，麦考瑞大学）；Department of Broadband Communication, Peng Cheng Laboratory（宽带通信部，鹏城实验室）；China Telecom, Sichuan Branch（中国电信四川分公司）
Federated Learning (FL) has opened the opportunity for collaboratively training machine learning models on heterogeneous mobile or Edge devices while keeping local data private.With an increase in its adoption, a growing concern is related to its economic and environmental cost (as is also the case for other machine learning techniques).Unfortunately, little work has been done to optimize its energy consumption or emissions of carbon dioxide or equivalents, as energy minimization is usually left as a secondary objective.In this paper, we investigate the problem of minimizing the energy consump…
FedFiTS: Fitness-Selected, Slotted Client Scheduling for Trustworthy Federated Learning in Healthcare AI
Department of Computer Science University of the Western Cape（计算机科学系，西开普敦大学）；Department of Computer Science Missouri University of Science and Technology（计算机科学系，密苏里科学与技术大学）
WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork Scheduling
University of Houston（休斯顿大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Peng Cheng Laboratory（鹏城实验室）；Stevens Institute of Technology（史蒂文斯理工学院）
Singapore University of Design and Technology（新加坡科技设计大学）；Shenzhen University（深圳大学）；East China Normal University（华东师范大学）；Zhejiang University（浙江大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Personalized Federated Learning (PFL) is a new Federated Learning (FL) approach to address the heterogeneity issue of the datasets generated by distributed user equipments (UEs). However, most existing PFL implementations rely on synchronous training to ensure good convergence performances, which may lead to a serious straggler problem, where the training time is heavily prolonged by the slowest UE. To address this issue, we propose a semi-synchronous PFL algorithm, termed as Semi-Synchronous Personalized FederatedAveraging (PerFedS$^2$), over mobile edge networks. By jointly optimizing the wi…
Over-the-Air Federated Averaging with Limited Power and Privacy Budgets
Queen Mary University of London（伦敦玛丽女王大学）；Brunel University London（伦敦布鲁内尔大学）；Southeast University（东南大学）；Hainan University（海南大学）；Nanjing University of Science and Technology（南京理工大学）；University of Kent（肯特大学）
Hong Kong Baptist University（香港浸会大学）；FedML Inc.（FedML公司）；Inha University（仁荷大学）；The Hong Kong University of Science and Technology（香港科技大学）
Federated Learning (FL) enables collaborations among clients for train machine learning models while protecting their data privacy. Existing FL simulation platforms that are designed from the perspectives of traditional distributed training, suffer from laborious code migration between simulation and production, low efficiency, low GPU utility, low scalability with high hardware requirements and difficulty of simulating stateful clients. In this work, we firstly demystify the challenges and bottlenecks of simulating FL, and design a new FL system named as FedML \texttt{Parrot}. It improves the…
