Motivated by the increasing computational capacity of wireless user equipments (UEs), e.g., smart phones, tablets, or vehicles, as well as the increasing concerns about sharing private data, a new machine learning model has emerged, namely federated learning (FL), that allows a decoupling of data acquisition and computation at the central unit. Unlike centralized learning taking place in a data center, FL usually operates in a wireless edge network where the communication medium is resource-constrained and unreliable. Due to limited bandwidth, only a portion of UEs can be scheduled for updates at each iteration. Due to the shared nature of the wireless medium, transmissions are subjected to interference and are not guaranteed. The performance of FL system in such a setting is not well understood. In this paper, an analytical model is developed to characterize the performance of FL in wireless networks. Particularly, tractable expressions are derived for the convergence rate of FL in a wireless setting, accounting for effects from both scheduling schemes and inter-cell interference. Using the developed analysis, the effectiveness of three different scheduling policies, i.e., random scheduling (RS), round robin (RR), and proportional fair (PF), are compared in terms of FL convergence rate. It is shown that running FL with PF outperforms RS and RR if the network is operating under a high signal-to-interference-plus-noise ratio (SINR) threshold, while RR is more preferable when the SINR threshold is low. Moreover, the FL convergence rate decreases rapidly as the SINR threshold increases, thus confirming the importance of compression and quantization of the update parameters. The analysis also reveals a trade-off between the number of scheduled UEs and subchannel bandwidth under a fixed amount of available spectrum.
Federated learning (FL) is a machine learning model that preserves data privacy in the training process. Specifically, FL brings the model directly to the user equipments (UEs) for local training, where an edge server periodically collects the trained parameters to produce an improved model and sends it back to the UEs. However, since communication usually occurs through a limited spectrum, only a portion of the UEs can update their parameters upon each global aggregation. As such, new scheduling algorithms have to be engineered to facilitate the full implementation of FL. In this paper, based…
Federated Learning (FL) is a newly emerged decentralized machine learning (ML) framework that combines on-device local training with server-based model synchronization to train a centralized ML model over distributed nodes. In this paper, we propose an asynchronous FL framework with periodic aggregation to eliminate the straggler issue in FL systems. For the proposed model, we investigate several device scheduling and update aggregation policies and compare their performances when the devices have heterogeneous computation capabilities and training data distributions. From the simulation resul…
Hitachi (China) Research & Development Cooperation（日立（中国）研发有限公司）
In federated learning (FL), devices contribute to the global training by uploading their local model updates via wireless channels. Due to limited computation and communication resources, device scheduling is crucial to the convergence rate of FL. In this paper, we propose a joint device scheduling and resource allocation policy to maximize the model accuracy within a given total training time budget for latency constrained wireless FL. A lower bound on the reciprocal of the training performance loss, in terms of the number of training rounds and the number of scheduled devices per round, is d…
Zhejiang University（浙江大学）；Northwestern University（西北大学）；The University of Hong Kong（香港大学）
In cellular federated edge learning (FEEL), multiple edge devices holding local data jointly train a neural network by communicating learning updates with an access point without exchanging their data samples. With very limited communication resources, it is beneficial to schedule the most informative local learning updates. In this paper, a novel scheduling policy is proposed to exploit both diversity in multiuser channels and diversity in the "importance" of the edge devices' learning updates. First, a new probabilistic scheduling framework is developed to yield unbiased update aggregation i…
Federated learning (FL) in a bandwidth-limited network with energy-limited user equipments (UEs) is under-explored. In this paper, to jointly save energy consumed by the battery-limited UEs and accelerate the convergence of the global model in FL for the bandwidth-limited network, we propose the sliding differential evolution-based scheduling (SDES) policy. To this end, we first formulate an optimization that aims to minimize a weighted sum of energy consumption and model training convergence. Then, we apply the SDES with parallel differential evolution (DE) operations in several small-scale w…
This paper develops a class of low-complexity device scheduling algorithms for over-the-air federated learning via the method of matching pursuit. The proposed scheme tracks closely the close-to-optimal performance achieved by difference-of-convex programming, and outperforms significantly the well-known benchmark algorithms based on convex relaxation. Compared to the state-of-the-art, the proposed scheme poses a drastically lower computational load on the system: For $K$ devices and $N$ antennas at the parameter server, the benchmark complexity scales with $\left(N^2+K\right)^3 + N^6$ while t…
The Age-of-Information (AoI) is a new performance metric recently proposed for measuring the freshness of information in information-update systems. In this work, we conduct a systematic and comparative study to investigate the impact of scheduling policies on the AoI performance in single-server queues and provide useful guidelines for the design of AoI-efficient scheduling policies. Specifically, we first perform extensive simulations to demonstrate that the update-size information can be leveraged for achieving a substantially improved AoI compared to non-size-based (or arrival-time-based) …
In this paper, we study the joint routing-scheduling problem in energy harvesting communication networks. Our policies, which are based on stochastic subgradient methods on the dual domain, act as an energy harvesting variant of the stochastic family of backpresure algorithms. Specifically, we propose two policies: (i) the Stochastic Backpressure with Energy Harvesting (SBP-EH), in which a node's routing-scheduling decisions are determined by the difference between the Lagrange multipliers associated to their queue stability constraints and their neighbors'; and (ii) the Stochastic Soft Backpr…
Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Tsinghua University（清华大学）；Imperial College London（伦敦帝国学院）
Machine learning and wireless communication technologies are jointly facilitating an intelligent edge, where federated edge learning (FEEL) is a promising training framework. As wireless devices involved in FEEL are resource limited in terms of communication bandwidth, computing power and battery capacity, it is important to carefully schedule them to optimize the training performance. In this work, we consider an over-the-air FEEL system with analog gradient aggregation, and propose an energy-aware dynamic device scheduling algorithm to optimize the training performance under energy constrain…
We consider a wireless broadcast network with a base station sending time-sensitive information to a number of clients through unreliable channels. The Age of Information (AoI), namely the amount of time that elapsed since the most recently delivered packet was generated, captures the freshness of the information. We formulate a discrete-time decision problem to find a transmission scheduling policy that minimizes the expected weighted sum AoI of the clients in the network. We first show that in symmetric networks a Greedy policy, which transmits the packet with highest current age, is optimal…
In this paper, we study asynchronous federated learning (FL) in a wireless distributed learning network (WDLN). To allow each edge device to use its local data more efficiently via asynchronous FL, transmission scheduling in the WDLN for asynchronous FL should be carefully determined considering system uncertainties, such as time-varying channel and stochastic data arrivals, and the scarce radio resources in the WDLN. To address this, we propose a metric, called an effectivity score, which represents the amount of learning from asynchronous FL. We then formulate an Asynchronous Learning-aware …
A point-to-point discrete-time scheduling problem of transmitting $B$ information bits within $T$ hard delay deadline slots is considered assuming that the underlying energy-bit cost function is a convex monomial. The scheduling objective is to minimize the expected energy expenditure while satisfying the deadline constraint based on information about the unserved bits, channel state/statistics, and the remaining time slots to the deadline. At each time slot, the scheduling decision is made without knowledge of future channel state, and thus there is a tension between serving many bits when th…
Non-Coherent Over-the-Air Federated Learning: Protocol, Convergence, and Device Scheduling
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Arizona State University（亚利桑那州立大学）；Northeastern University London（伦敦东北大学）；Shanghai HKU Education Center（上海港大教育中心）
Many data applications in the next generation cellular networks, such as content precaching and video progressive downloading, require flow-level quality of service (QoS) guarantees. One such requirement is deadline, where the transmission task needs to be completed before the application-specific time. To minimize the number of uncompleted transmission tasks, we study laxity-based scheduling policies in this paper. We propose a Less-Laxity-Higher-Possible-Rate (L$^2$HPR) policy and prove its asymptotic optimality in underloaded identical-deadline systems. The asymptotic optimality of L$^2$HPR…
We consider the resource allocation problem in cellular networks which support Device-to-Device Communications (D2D). For systems that enable D2D via only orthogonal resource sharing, we propose and analyze two resource allocation policies that guarantee access fairness among all users, while taking advantage of multi-user diversity and local D2D communications, to provide marked improvements over existing cellular-only policies. The first policy, the Cellular Fairness Scheduling (CFS) Policy, provides the simplest D2D extension to existing cellular systems, while the second policy, the D2D Fa…
In this paper, the deployment of federated learning (FL) is investigated in an energy harvesting wireless network in which the base station (BS) employs massive multiple-input multiple-output (MIMO) to serve a set of users powered by independent energy harvesting sources. Since a certain number of users may not be able to participate in FL due to the interference and energy constraints, a joint energy management and user scheduling problem in FL over wireless systems is formulated. This problem is formulated as an optimization problem whose goal is to minimize the FL training loss via optimizi…
Owing to the increasing need for massive data analysis and model training at the network edge, as well as the rising concerns about the data privacy, a new distributed training framework called federated learning (FL) has emerged. In each iteration of FL (called round), the edge devices update local models based on their own data and contribute to the global training by uploading the model updates via wireless channels. Due to the limited spectrum resources, only a portion of the devices can be scheduled in each round. While most of the existing work on scheduling focuses on the convergence of…
Princeton University（普林斯顿大学）；Imperial College London（帝国理工学院）
We study federated learning (FL) at the wireless edge, where power-limited devices with local datasets collaboratively train a joint model with the help of a remote parameter server (PS). We assume that the devices are connected to the PS through a bandwidth-limited shared wireless channel. At each iteration of FL, a subset of the devices are scheduled to transmit their local model updates to the PS over orthogonal channel resources, while each participating device must compress its model update to accommodate to its link capacity. We design novel scheduling and resource allocation policies th…
Feature-Based Semantics-Aware Scheduling for Energy-Harvesting Federated Learning
Department of Computer and Information Science（计算机与信息科学系）；Linköping University（林奈技术大学）；Department of Information Engineering（信息工程系）；University of Brescia（布雷西亚大学）；ZJU-UIUC Institute（浙大-伊利诺伊大学联合研究所）；Zhejiang University（浙江大学）
The problem of designing efficient feedback-based scheduling policies for chunked codes (CC) over packet networks with delay and loss is considered. For networks with feedback, two scheduling policies, referred to as random push (RP) and local-rarest-first (LRF), already exist. We propose a new scheduling policy, referred to as minimum-distance-first (MDF), based on the expected number of innovative successful packet transmissions at each node of the network prior to the "next" transmission time, given the feedback information from the downstream node(s) about the received packets. Unlike the …
Shenzhen Research Institute of Big Data（深圳大数据研究院）；Southern University of Science and Technology（南方科技大学）；China Academy of Information and Communications Technology（中国信息通信研究院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Zhejiang University（浙江大学）
The popular federated edge learning (FEEL) framework allows privacy-preserving collaborative model training via frequent learning-updates exchange between edge devices and server. Due to the constrained bandwidth, only a subset of devices can upload their updates at each communication round. This has led to an active research area in FEEL studying the optimal device scheduling policy for minimizing communication time. However, owing to the difficulty in quantifying the exact communication time, prior work in this area can only tackle the problem partially by considering either the communicatio…
In this paper, we study the scheduling problem for downlink transmission in a multi-channel (e.g., OFDM-based) wireless network. We focus on a single cell, with the aim of developing a unifying framework for designing low-complexity scheduling policies that can provide optimal performance in terms of both throughput and delay. We develop new easy-to-verify sufficient conditions for rate-function delay optimality (in the many-channel many-user asymptotic regime) and throughput optimality (in general non-asymptotic setting), respectively. The sufficient conditions allow us to prove rate-function…
School of Science and Engineering, the Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）理工学院）；Department of Electrical and Electronic Engineering, Imperial College London（伦敦帝国理工学院电气与电子工程系）
We consider federated edge learning (FEEL) over wireless fading channels taking into account the downlink and uplink channel latencies, and the random computation delays at the clients. We speed up the training process by overlapping the communication with computation. With fountain coded transmission of the global model update, clients receive the global model asynchronously, and start performing local computations right away. Then, we propose a dynamic client scheduling policy, called MRTP, for uploading local model updates to the parameter server (PS), which, at any time, schedules the clie…
A hard-deadline, opportunistic scheduling problem in which $B$ bits must be transmitted within $T$ time-slots over a time-varying channel is studied: the transmitter must decide how many bits to serve in each slot based on knowledge of the current channel but without knowledge of the channel in future slots, with the objective of minimizing expected transmission energy. In order to focus on the effects of delay and fading, we assume that no other packets are scheduled simultaneously and no outage is considered. We also assume that the scheduler can transmit at capacity where the underlying noi…
Virginia Tech（弗吉尼亚理工大学）；University of Colorado Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）；Ericsson Research（爱立信研究院）；Princeton University（普林斯顿大学）；University of Oulu（奥卢大学）
Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks ranging from coordinated trajectory planning to cooperative target recognition. However, due to the lack of continuous connections between the UAV swarm and ground base stations (BSs), using centralized ML will be challenging, particularly when dealing with a large volume of data. In this paper, a novel framework is proposed to implement distributed federated learning (FL) algorithms within a UAV swarm that consists of a leading UAV and several following UAVs. Each following UAV trains a l…
Singapore University of Technology and Design（新加坡科技设计大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
By exploiting the computing power and local data of distributed clients, federated learning (FL) features ubiquitous properties such as reduction of communication overhead and preserving data privacy. In each communication round of FL, the clients update local models based on their own data and upload their local updates via wireless channels. However, latency caused by hundreds to thousands of communication rounds remains a bottleneck in FL. To minimize the training latency, this work provides a multi-armed bandit-based framework for online client scheduling (CS) in FL without knowing wireles…
We study the performance of non-adaptive scheduling policies in computing systems with multiple servers. Compute jobs are mostly regular, with modest service requirements. However, there are sporadic data intensive jobs, whose expected service time is much higher than that of the regular jobs. Forthis model, we are interested in the effect of scheduling policieson the average time a job spends in the system. To this end, we introduce two performance indicators in a simplified, only-arrival system. We believe that these performance indicators are good predictors of the relative performance of t…
We consider the problem of joint link scheduling and power control for wireless networks with average transmission power constraints. Due to the high computational complexity of the optimal policies, we extend the class of greedy link scheduling policies to handle average power constraints. We develop a greedy link scheduling and power control scheme GECS, with provable performance guarantees. We show that the performance of our greedy scheduler can be characterized using the Local Pooling Factor (LPF) of a network graph, which has been previously used to characterize the stability of the Gree…
Dynamic Scheduling for Vehicle-to-Vehicle Communications Enhanced Federated Learning
Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University（北京信息科学与技术国家研究中心，电子工程系，清华大学）；School of Electronic and Information Engineering, Beijing Jiaotong University（电子信息工程学院，北京交通大学）
Army Research Laboratory（陆军研究实验室）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；University of Utah（犹他大学）
Federated learning (FL) is a useful tool in distributed machine learning that utilizes users' local datasets in a privacy-preserving manner. When deploying FL in a constrained wireless environment; however, training models in a time-efficient manner can be a challenging task due to intermittent connectivity of devices, heterogeneous connection quality, and non-i.i.d. data. In this paper, we provide a novel convergence analysis of non-convex loss functions using FL on both i.i.d. and non-i.i.d. datasets with arbitrary device selection probabilities for each round. Then, using the derived conver…
Distributed computing systems implement redundancy to reduce the job completion time and variability. Despite a large body of work about computing redundancy, the analytical performance evaluation of redundancy techniques in queuing systems is still an open problem. In this work, we take one step forward to analyze the performance of scheduling policies in systems with redundancy. In particular, we study the pattern of shared servers among replicas of different jobs. To this end, we employ combinatorics and graph theory and define and derive performance indicators using the statistics of the o…
We propose a variant of the Age of Information (AoI) metric called Channel-Aware Age of Information (CA-AoI). Unlike AoI, CA-AoI takes into account the channel conditions between the source and the intended destination to compute the "age" of the recent most update received by the destination. This new metric ensures that the resource allocation is not heavily tilted towards the sources with poor channel conditions. We design scheduling policies for multi-sensor systems in which sensors report their measurements to a central monitoring station via a shared unreliable communication channel with…
