Cooperative Learning via Federated Distillation over Fading Channels
Jin-Hyun Ahn, Osvaldo Simeone, Joonhyuk Kang
备注：
to appear in Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2020. arXiv admin note: text overlap with arXiv:1907.02745
Cooperative training methods for distributed machine learning are typically based on the exchange of local gradients or local model parameters. The latter approach is known as Federated Learning (FL). An alternative solution with reduced communication overhead, referred to as Federated Distillation (FD), was recently proposed that exchanges only averaged model outputs. While prior work studied implementations of FL over wireless fading channels, here we propose wireless protocols for FD and for an enhanced version thereof that leverages an offline communication phase to communicate ``mixed-up'' covariate vectors. The proposed implementations consist of different combinations of digital schemes based on separate source-channel coding and of over-the-air computing strategies based on analog joint source-channel coding. It is shown that the enhanced version FD has the potential to significantly outperform FL in the presence of limited spectral resources.
Wireless communications is often subject to channel fading. Various statistical models have been proposed to capture the inherent randomness in fading, and conventional model-based receiver designs rely on accurate knowledge of this underlying distribution, which, in practice, may be complex and intractable. In this work, we propose a neural network-based symbol detection technique for downlink fading channels, which is based on the maximum a-posteriori probability (MAP) detector. To enable training on a diverse ensemble of fading realizations, we propose a federated training scheme, in which …
In this study, a cross-layer design which combines adaptive modulation and coding (AMC) and hybrid automatic repeat request (HARQ) techniques for a cooperative wireless network is investigated analytically. Previous analyses of such systems in the literature are confined to the case where the fading channel is independent at each retransmission, which can be unrealistic unless the channel is varying very fast. On the other hand, temporal channel correlation can have a significant impact on the performance of HARQ systems. In this study, utilizing a Markov channel model which accounts for the t…
Non-orthogonal Multiple Access (NOMA) has become a salient technology for improving the spectral efficiency of the next generation 5G wireless communication networks. In this paper, the achievable average rate of an Opportunistic Non-Orthogonal Multiple Access (O-NOMA) based Cooperative Relaying System (CRS) is studied under Rician fading channels with Channel State Information (CSI) available at the source terminal. Based on CSI, for opportunistic transmission, the source immediately chooses either the direct transmission or the cooperative NOMA transmission using the relay, which can provide…
Cooperative vehicular networks will play a vital role in the coming years to implement various intelligent transportation-related applications. Both vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications will be needed to reliably disseminate information in a vehicular network. In this regard, a roadside unit (RSU) equipped with multiple antennas can improve the network capacity. While the traditional approaches assume antennas to experience independent fading, we consider a more practical uplink scenario where antennas at the RSU experience correlated fading. In particula…
German Research Centre for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；University of Kaiserslautern（凯泽斯劳滕大学）
Opportunistic relay selection (ORS) has been recognized as a simple but efficient method for mobile nodes to achieve cooperative diversity in slow fading channels. However, the wrong selection of the best relay arising from outdated channel state information (CSI) in fast time-varying channels substantially degrades its performance. With the proliferation of high-mobility applications and the adoption of higher frequency bands in 5G and beyond systems, the problem of outdated CSI will become more serious. Therefore, the design of a novel cooperative method that is applicable to not only slow f…
This letter proposes the usage of orbital angular momentum (OAM) for cooperative non-orthogonal multiple access (CNOMA) to enhance sum capacity (SC) for the future cellular communication system. The proposed CNOMA-OAM scheme is analyzed and compared with other schemes, i.e., conventional CNOMA, conventional orthogonal multiple access (OMA) with OAM. The impact of the power allocation factor for OAM beam over SC is also analyzed. The analytical result is justified by simulation results which demonstrate that the proposed CNOMA-OAM provides higher SC compared to other schemes.
Energy-harvesting and wireless power transfer in cooperative relaying networks have recently attracted a considerable amount of research attention. Most of the existing work on this topic however focuses on Rayleigh fading channels which represents outdoor environments. Unlike these studies, in this paper we analyze the performance of wireless power transfer in two-hop decode-and-forward (DF) cooperative relaying systems in indoor channels characterized by log-normal fading. Three well-known energy-harvesting protocols are considered in our evaluations: a) time-switching relaying (TSR), b) pow…
Visible light positioning (VLP) has drawn plenty of attention as a promising indoor positioning technique. However, in nonstationary environments, the performance of VLP is limited because of the highly time-varying channels. To improve the positioning accuracy and generalization capability in nonstationary environments, a cooperative VLP scheme based on federated learning (FL) is proposed in this paper. Exploiting the FL framework, a global model adaptive to environmental changes can be jointly trained by users without sharing private data of users. Moreover, a Cooperative Visible-light Posit…
Over-the-air federated learning (OTA-FL) has emerged as an efficient mechanism that exploits the superposition property of the wireless medium and performs model aggregation for federated learning in the air. OTA-FL is naturally sensitive to wireless channel fading, which could significantly diminish its learning accuracy. To address this challenge, in this paper, we propose an OTA-FL algorithm called CHARLES (channel-quality-aware over-the-air local estimating and scaling). Our CHARLES algorithm performs channel state information (CSI) estimation and adaptive scaling to mitigate the impacts o…
Wireless communication faces adversities due to noise, fading, and path loss. Multiple-Input Multiple-Output (MIMO) systems are used to overcome individual fading effect by employing transmit diversity. Duo to user single-antenna, Cooperation between at least two users is able to provide spatial diversity. This paper presents the evaluation of the performances of the Amplify and Forward (AF) cooperative system for different relay positions using several network topologies over Rayleigh and Rician fading channel. Furthermore, we present the performances of AF cooperative system with various pow…
In this paper, we propose a novel cooperative multi-relay transmission scheme for mobile terminals to exploit spatial diversity. By improving the timeliness of measured channel state information (CSI) through deep learning (DL)-based channel prediction, the proposed scheme remarkably lowers the probability of wrong relay selection arising from outdated CSI in fast time-varying channels. It inherits the simplicity of opportunistic relaying by selecting a single relay, avoiding the complexity of multi-relay coordination and synchronization. Numerical results reveal that it can achieve full diver…
This paper studies the performance of a downlink non-orthogonal multiple access (NOMA) based cooperative network with maximal ratio transmission/receive antenna selection (MRT/RAS) over Nakagami-m fading channels in the presence of channel estimation errors (CEEs). In the system, a base station communicates with multiple mobile users through a half duplex channel state information based amplify-and-forward relay. All nodes are equipped with multiple antennas and the hybrid diversity technique MRT/RAS is employed in both hops. The outage behavior of the system is investigated by driving closed-…
In this work, we consider the performance analysis of the uplink cellular networks with free space optics (FSO) backhauling. The user equipment (UE) communicates with the nearest Base Station (BS) in first slot while in second slot, the BS converts the received radio frequency (RF) signal into FSO pulse and transmits to the data center. We adopt the Rayleigh fading for the uplink channels while the FSO backhaul encompasses the turbulence-induced fading which follows the Málaga distribution, the weather pathloss, and the pointing errors fading which is distributed following the generalized Beck…
In this paper, we consider federated learning (FL) over a noisy fading multiple access channel (MAC), where an edge server aggregates the local models transmitted by multiple end devices through over-the-air computation (AirComp). To realize efficient analog federated learning over wireless channels, we propose an AirComp-based FedSplit algorithm, where a threshold-based device selection scheme is adopted to achieve reliable local model uploading. In particular, we analyze the performance of the proposed algorithm and prove that the proposed algorithm linearly converges to the optimal solution…
In this study, we propose an over-the-air computation (AirComp) scheme for federated edge learning (FEEL) without channel state information (CSI) at the edge devices (EDs) or the edge server (ES). The proposed scheme relies on non-coherent communication techniques for achieving distributed training by majority vote (MV). In this work, the votes, i.e., the signs of the local gradients, from the EDs are represented with the pulse-position modulation (PPM) symbols constructed with discrete Fourier transform (DFT)-spread orthogonal frequency division multiplexing (OFDM) (DFT-s-OFDM). By taking the…
Federated learning (FL) has recently emerged as a promising technology to enable artificial intelligence (AI) at the network edge, where distributed mobile devices collaboratively train a shared AI model under the coordination of an edge server. To significantly improve the communication efficiency of FL, over-the-air computation allows a large number of mobile devices to concurrently upload their local models by exploiting the superposition property of wireless multi-access channels. Due to wireless channel fading, the model aggregation error at the edge server is dominated by the weakest cha…
The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；SRIBD（深圳大数据研究院）；Wuhan University（武汉大学）
In this overview paper, data-driven learning model-based cooperative localization and location data processing are considered, in line with the emerging machine learning and big data methods. We first review (1) state-of-the-art algorithms in the context of federated learning, (2) two widely used learning models, namely the deep neural network model and the Gaussian process model, and (3) various distributed model hyper-parameter optimization schemes. Then, we demonstrate various practical use cases that are summarized from a mixture of standard, newly published, and unpublished works, which c…
In this paper, cooperative edge caching problem is studied in fog radio access networks (F-RANs). Given the non-deterministic polynomial hard (NP-hard) property of the problem, a dueling deep Q network (Dueling DQN) based caching update algorithm is proposed to make an optimal caching decision by learning the dynamic network environment. In order to protect user data privacy and solve the problem of slow convergence of the single deep reinforcement learning (DRL) model training, we propose a federated reinforcement learning method with quantization (FRLQ) to implement cooperative training of m…
Federated Learning (FL), an emerging paradigm for fast intelligent acquisition at the network edge, enables joint training of a machine learning model over distributed data sets and computing resources with limited disclosure of local data. Communication is a critical enabler of large-scale FL due to significant amount of model information exchanged among edge devices. In this paper, we consider a network of wireless devices sharing a common fading wireless channel for the deployment of FL. Each device holds a generally distinct training set, and communication typically takes place in a Device…
Over-the-air federated edge learning (Air-FEEL) is a communication-efficient solution for privacy-preserving distributed learning over wireless networks. Air-FEEL allows "one-shot" over-the-air aggregation of gradient/model-updates by exploiting the waveform superposition property of wireless channels, and thus promises an extremely low aggregation latency that is independent of the network size. However, such communication efficiency may come at a cost of learning performance degradation due to the aggregation error caused by the non-uniform channel fading over devices and noise perturbation.…
In this work, we study the performance of a single relay Multiple Input Multiple Output (MIMO) cooperative communication system based on Decode and Forward (DF) relaying protocol for two strategies using transmit antenna selection at the source. The first strategy uses one antenna between the relay and the destination, and the second strategy uses Space Time Block Coding (STBC). All channels follow the Rayleigh fading distribution. We derive the expression and Upper Bound for Symbol Error Rate (SER) for M-ary Phase Shift Keying (M-PSK), and the diversity order for both strategies. The analytic…
In this paper, we consider the problem of wireless federated learning based on sign stochastic gradient descent (signSGD) algorithm via a multiple access channel. When sending locally computed gradient's sign information, each mobile device requires to apply precoding to circumvent wireless fading effects. In practice, however, acquiring perfect knowledge of channel state information (CSI) at all mobile devices is infeasible. In this paper, we present a simple yet effective precoding method with limited channel knowledge, called sign-alignment precoding. The idea of sign-alignment precoding is…
In this paper, we provide a global framework analysis of a dual-hop mixed Radio Frequency (RF)/Free Space Optical (FSO) system with multiple branches/relays wherein the first and second hops, respectively, consist of RF and FSO channels. To cover various cases of fading, we propose generalized channels' models for RF and FSO links that follow the Nakagami-m and the Double Generalized Gamma (DGG) distributions, respectively. Moreover, we suggest Channel State Information (CSI)-assisted relaying or variable relaying gain based Amplifiy-and-Forward (AF) amplification. Partial relay selection with…
Federated learning (FL) is a collaborative machine learning paradigm, which enables deep learning model training over a large volume of decentralized data residing in mobile devices without accessing clients' private data. Driven by the ever increasing demand for model training of mobile applications or devices, a vast majority of FL tasks are implemented over wireless fading channels. Due to the time-varying nature of wireless channels, however, random delay occurs in both the uplink and downlink transmissions of FL. How to analyze the overall time consumption of a wireless FL task, or more s…
Princeton University（普林斯顿大学）；Bilkent University（比尔肯特大学）；Imperial College London（伦敦帝国理工学院）
We study federated edge learning (FEEL), where wireless edge devices, each with its own dataset, learn a global model collaboratively with the help of a wireless access point acting as the parameter server (PS). At each iteration, wireless devices perform local updates using their local data and the most recent global model received from the PS, and send their local updates to the PS over a wireless fading multiple access channel (MAC). The PS then updates the global model according to the signal received over the wireless MAC, and shares it with the devices. Motivated by the additive nature o…
This paper analyzes the performance of a cooperative relaying non-orthogonal multiple access (NOMA) network over Fisher-Snedecor F composite fading channels. Specifically, one base station (BS) is assumed to communicate with two receiving mobile users with one acting also as a decode-and-forward (DF) relay. To highlight the achievable performance gains of the NOMA scheme, conventional relaying with orthogonal multiple access (OMA) is also analyzed. For the two systems under consideration, we derive novel exact closed-form expressions for the ergodic capacity along with the corresponding asympt…
In this paper, the integration of cooperative non-orthogonal multiple access (CNOMA) and power splitting based simultaneous wireless information and power transfer (SWIPT) is proposed to improve the spectral and energy efficiency. Moreover, different orbital angular momentum (OAM) based signals are also considered in this paper to transmit additional symbols to cell center user (CCU) and cell edge user (CEU). In the proposed technique, CCU is used as energy-constrained relay to forward signal to CEU. CCU harvests energy from the transmitted signal by the BS using the power splitting technique.…
