A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems
POSTECH（浦项工科大学）；Princeton University（普林斯顿大学）；Nanjing University of Science and Technology（南京理工大学）
Yo-Seb Jeon, Mohammad Mohammadi Amiri, Jun Li, H. Vincent Poor
备注：
The title of the paper has been changed from "Gradient Estimation for Federated Learning over Massive MIMO Communication Systems" to "A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems"
Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices, each with its own local training data set. In this paper, we present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems in which the central server equipped with a massive antenna array communicates with the wireless devices. One major challenge in system design is to reconstruct local gradient vectors accurately at the central server, which are computed-and-sent from the wireless devices. To overcome this challenge, we first establish a transmission strategy to construct sparse transmitted signals from the local gradient vectors at the devices. We then propose a compressive sensing algorithm enabling the server to iteratively find the linear minimum-mean-square-error (LMMSE) estimate of the transmitted signal by exploiting its sparsity. We also derive an analytical threshold for the residual error at each iteration, to design the stopping criterion of the proposed algorithm. We show that for a sparse transmitted signal, the proposed algorithm requires less computationally complexity than LMMSE. Simulation results demonstrate that the presented approach outperforms conventional linear beamforming approaches and reduces the performance gap between federated learning and centralized learning with perfect reconstruction.
Federated learning (FL) is a type of distributed machine learning at the wireless edge that preserves the privacy of clients' data from adversaries and even the central server. Existing federated learning approaches either use (i) secure multiparty computation (SMC) which is vulnerable to inference or (ii) differential privacy which may decrease the test accuracy given a large number of parties with relatively small amounts of data each. To tackle the problem with the existing methods in the literature, In this paper, we introduce incorporate federated learning in the inner-working of MIMO sys…
United States CCDC Army Research Laboratory（美国CCDC陆军研究实验室）
Millimeter wave multiple-input multiple-output (mmWave-MIMO) systems with small number of radio-frequency (RF) chains have limited multiplexing gain. Spatial path index modulation (SPIM) is helpful in improving this gain by utilizing additional signal bits modulated by the indices of spatial paths. In this paper, we introduce model-based and model-free frameworks for beamformer design in multi-user SPIM-MIMO systems. We first design the beamformers via model-based manifold optimization algorithm. Then, we leverage federated learning (FL) with dropout learning (DL) to train a learning model on …
University of Virginia（弗吉尼亚大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；Princeton University（普林斯顿大学）
We propose a novel communication design, termed random orthogonalization, for federated learning (FL) in a massive multiple-input and multiple-output (MIMO) wireless system. The key novelty of random orthogonalization comes from the tight coupling of FL and two unique characteristics of massive MIMO -- channel hardening and favorable propagation. As a result, random orthogonalization can achieve natural over-the-air model aggregation without requiring transmitter side channel state information (CSI) for the uplink phase of FL, while significantly reducing the channel estimation overhead at the…
Machine learning for hybrid beamforming has been extensively studied by using centralized machine learning (CML) techniques, which require the training of a global model with a large dataset collected from the users. However, the transmission of the whole dataset between the users and the base station (BS) is computationally prohibitive due to limited communication bandwidth and privacy concerns. In this work, we introduce a federated learning (FL) based framework for hybrid beamforming, where the model training is performed at the BS by collecting only the gradients from the users. We design …
For distributed learning among collaborative users, this paper develops and analyzes a communication-efficient scheme for federated learning (FL) over the air, which incorporates 1-bit compressive sensing (CS) into analog aggregation transmissions. To facilitate design parameter optimization, we theoretically analyze the efficacy of the proposed scheme by deriving a closed-form expression for the expected convergence rate of the FL over the air. Our theoretical results reveal the tradeoff between convergence performance and communication efficiency as a result of the aggregation errors caused …
With the explosive growth of data and wireless devices, federated learning (FL) over wireless medium has emerged as a promising technology for large-scale distributed intelligent systems. Yet, the urgent demand for ubiquitous intelligence will generate a large number of concurrent FL tasks, which may seriously aggravate the scarcity of communication resources. By exploiting the analog superposition of electromagnetic waves, over-the-air computation (AirComp) is an appealing solution to alleviate the burden of communication required by FL. However, sharing frequency-time resources in over-the-a…
Duzce University（杜兹杰大学）；University of Hertfordshire（赫特福德大学）；Koc University（科奇大学）
Machine learning (ML) has attracted a great research interest for physical layer design problems, such as channel estimation, thanks to its low complexity and robustness. Channel estimation via ML requires model training on a dataset, which usually includes the received pilot signals as input and channel data as output. In previous works, model training is mostly done via centralized learning (CL), where the whole training dataset is collected from the users at the base station (BS). This approach introduces huge communication overhead for data collection. In this paper, to address this challe…
Queen’s University Belfast（贝尔法斯特女王大学）；New York University（纽约大学）
Federated learning (FL) has been considered as a promising learning framework for future machine learning systems due to its privacy preservation and communication efficiency. In beyond-5G/6G systems, it is likely to have multiple FL groups with different learning purposes. This scenario leads to a question: How does a wireless network support multiple FL groups? As an answer, we first propose to use a cell-free massive multiple-input multiple-output (MIMO) network to guarantee the stable operation of multiple FL processes by letting the iterations of these FL processes be executed together wi…
Federated Learning allows distributed entities to train a common model collaboratively without sharing their own data. Although it prevents data collection and aggregation by exchanging only parameter updates, it remains vulnerable to various inference and reconstruction attacks where a malicious entity can learn private information about the participants' training data from the captured gradients. Differential Privacy is used to obtain theoretically sound privacy guarantees against such inference attacks by noising the exchanged update vectors. However, the added noise is proportional to the …
This paper considers the problem of estimating the sparse mmWave massive multiple input - multiple output (MIMO) OFDM channel from one-bit quantized measurements. Unlike previous grid-based one-bit compressive sensing approaches, we present a gridless convex method to recover sparse channel form one-bit measurements via Binary Atomic Norm Minimization (BiANM) and Reweighted Binary Atomic Norm Minimization (ReBiANM). Simulation results verify the accuracy of the binary and reweighted binary atomic norm minimization techniques.
School of Mathematics and Statistics, Beijing Jiaotong University（北京交通大学数学与统计学院）；Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London（帝国理工学院工程学院电气与电子工程系）
In colocated compressive sensing MIMO radar, the measurement matrix is specified by antenna placement. To guarantee an acceptable recovery performance, this measurement matrix should satisfy certain properties, e.g., a small coherence. Prior work in the literature often employs randomized placement algorithms which optimize the prior distribution of antenna locations. The performance of these algorithms is suboptimal, as they can be easily enhanced via expurgation. In this paper, we suggest an iterative antenna placement algorithm which determines the antenna locations deterministically. The p…
In the area of near-field millimeter-wave imaging, the generalized sparse array synthesis (SAS) method is in great demand. The traditional methods usually employ the greedy algorithms, which may have the convergence problem. This paper proposes a convex optimization model for the multiple-input multiple-output (MIMO) array design based on the compressive sensing (CS) approach. We generate a block shaped reference pattern, to be used as an optimizing target. The pattern occupies the entire imaging area of interest in order to involve the effect of each pixel into the optimization model. In MIMO…
Efficient channel estimation is challenging in full-dimensional multiple-input multiple-output communication systems, particularly in those with hybrid digital-analog architectures. Under a compressive sensing framework, this letter first designs a uniform dictionary based on a spherical Fibonacci grid to represent channels in a sparse domain, yielding smaller angular errors in three-dimensional beamspace than traditional dictionaries. Then, a Bayesian inference-aided greedy pursuit algorithm is developed to estimate channels in the frequency domain. Finally, simulation results demonstrate tha…
The large beamforming gain used to operate at millimeter wave (mmWave) frequencies requires obtaining channel information to configure hybrid antenna arrays. Previously proposed wideband channel estimation strategies, however, assume perfect time-frequency synchronization and neglect phase noise, making these approaches impractical. Consequently, achieving time-frequency synchronization between transmitter and receiver and correcting for phase noise (PN) as the channel is estimated, is the greatest challenge yet to be solved in order to configure hybrid precoders and combiners in practical set…
This paper considers the problem of estimating the sparse millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) OFDM channel from 1-bit quantized measurements. Unlike previous quantized on-grid approaches to compressive sensing, we introduce an off-grid convex scheme which uses binary atomic norm minimization (BiANM) to estimate sparse channel form 1-bit measurements. Moreover, in this paper, we propose a decoupled angle-delay version of BiANM called decoupled binary atomic norm minimization (DeBiANM) in order to reduce complexity. Further, for improving the accuracy of the es…
Reconfigurable intelligent surface (RIS) has been recognized as a potential technology for 5G beyond and attracted tremendous research attention. However, channel estimation in RIS-aided system is still a critical challenge due to the excessive amount of parameters in cascaded channel. The existing compressive sensing (CS)-based RIS estimation schemes only adopt incomplete sparsity, which induces redundant pilot consumption. In this paper, we exploit the specific triple-structured sparsity of the cascaded channel, i.e., the common column sparsity, structured row sparsity after offset compensat…
Synchronization is a fundamental procedure in cellular systems whereby an UE acquires the time and frequency information required to decode the data transmitted by a BS. Due to the necessity of using large antenna arrays to obtain the beamforming gain required to compensate for small antenna aperture, synchronization must be performed either jointly with beam training as in 5G NR, or at the low SNR regime if the high-dimensional mmWave MIMO channel is to be estimated. To circumvent this problem, this work proposes the first synchronization framework for mmWave MIMO that is robust to both TO, C…
We provide the optimal receive combining strategy for federated learning in multiple-input multiple-output (MIMO) systems. Our proposed algorithm allows the clients to perform individual gradient sparsification which greatly improves performance in scenarios with heterogeneous (non i.i.d.) training data. The proposed method beats the benchmark by a wide margin.
The doubly selective (DS) channel estimation in the large-scale multiple-input multiple-output (MIMO) systems is a challenging problem due to the large number of the channel coefficients to be estimated, which requires unaffordable and prohibitive pilot overhead. In this paper, firstly we conduct the analysis about the common sparsity of the basis expansion model (BEM) coefficients among all the BEM orders and all the transmit-receive antenna pairs. Then a novel pilot pattern is proposed, which inserts the guard pilots to deal with the inter carrier interference (ICI) under the superimposed pi…
Millimeter (mm) wave massive MIMO has the potential for delivering orders of magnitude increases in mobile data rates, with compact antenna arrays providing narrow steerable beams for unprecedented levels of spatial reuse. A fundamental technical bottleneck, however, is rapid spatial channel estimation and beam adaptation in the face of mobility and blockage. Recently proposed compressive techniques which exploit the sparsity of mm wave channels are a promising approach to this problem, with overhead scaling linearly with the number of dominant paths and logarithmically with the number of arra…
Given the high degree of computational complexity of the channel estimation technique based on the conventional one-dimensional (1-D) compressive sensing (CS) framework employed in the hybrid beamforming architecture, this study proposes two low-complexity channel estimation strategies. One is two-stage CS, which exploits row-group sparsity to estimate angle-of-arrival (AoA) first and uses the conventional 1-D CS method to obtain angle-of-departure (AoD). The other is two-dimensional (2-D) CS, which utilizes a 2-D dictionary to reconstruct the 2-D sparse signal. To conduct a meaningful compari…
Federated learning (FL) with its data privacy protection and communication efficiency has been considered as a promising learning framework for beyond-5G/6G systems. We consider a scenario where a group of downlink non-FL users are jointly served with a group of FL users using massive multiple-input multiple-output technology. The main challenge is how to utilise the resource to optimally serve both FL and non-FL users. We propose a communication scheme that serves the downlink of the non-FL users (UEs) and the uplink of FL UEs in each half of the frequency band. We formulate an optimization p…
We propose a novel uplink communication method, coined random orthogonalization, for federated learning (FL) in a massive multiple-input and multiple-output (MIMO) wireless system. The key novelty of random orthogonalization comes from the tight coupling of FL model aggregation and two unique characteristics of massive MIMO - channel hardening and favorable propagation. As a result, random orthogonalization can achieve natural over-the-air model aggregation without requiring transmitter side channel state information, while significantly reducing the channel estimation overhead at the receiver…
This paper considers massive access in massive multiple-input multiple-output (MIMO) systems and proposes an adaptive active user detection and channel estimation scheme based on compressive sensing. By exploiting the sporadic traffic of massive connected user equipments and the virtual angular domain sparsity of massive MIMO channels, the proposed scheme can support massive access with dramatically reduced access latency. Specifically, we design non-orthogonal pseudo-random pilots for uplink broadband massive access, and formulate the active user detection and channel estimation problems as a…
In this paper, we develop a dynamic detection network (DDNet) based detector for multiple-input multiple-output (MIMO) systems. By constructing an improved DetNet (IDetNet) detector and the OAMPNet detector as two independent network branches, the DDNet detector performs sample-wise dynamic routing to adaptively select a better one between the IDetNet and the OAMPNet detectors for every samples under different system conditions. To avoid the prohibitive transmission overhead of dataset collection in centralized learning (CL), we propose the federated averaging (FedAve)-DDNet detector, where al…
Federated learning (FL) is recognized as a key enabling technology to support distributed artificial intelligence (AI) services in future 6G. By supporting decentralized data training and collaborative model training among devices, FL inherently tames privacy leakage and reduces transmission costs. Whereas, the performance of the wireless FL is typically restricted by the communication latency. Multiple-input multiple-output (MIMO) technique is one promising solution to build up a communication-efficient edge FL system with limited radio resources. In this paper, we propose a novel joint devic…
While 5G networks are being rolled out, the definition of the potential 5G-Advanced features and the identification of disruptive technologies for 6G systems are being addressed by the scientific and academic communities to tackle the challenges that 2030 communication systems will face, such as terabit-capacity and always-on networks. In this framework, it is globally recognised that Non-Terrestrial Networks (NTN) will play a fundamental role in support to a fully connected world, in which physical, human, and digital domains will converge. In this framework, one of the main challenges that N…
Beijing Institute of Technology（北京理工大学）；CNRS（法国国家科学研究中心）；CentraleSupélec（中央理工-高等电力学院）；Université Paris-Saclay（巴黎-萨克雷大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Integrating large intelligent reflecting surfaces (IRS) into millimeter-wave (mmWave) massive multi-input-multi-ouput (MIMO) has been a promising approach for improved coverage and throughput. Most existing work assumes the ideal channel estimation, which can be challenging due to the high-dimensional cascaded MIMO channels and passive reflecting elements. Therefore, this paper proposes a deep denoising neural network assisted compressive channel estimation for mmWave IRS systems to reduce the training overhead. Specifically, we first introduce a hybrid passive/active IRS architecture, where v…
Channel estimation (CE) for millimeter-wave (mmWave) lens-array suffers from prohibitive training overhead, whereas the state-of-the-art solutions require an extra complicated radio frequency phase shift network. By contrast, lens-array using antenna switching network (ASN) simplifies the hardware, but the associated CE is a challenging task due to the constraint imposed by ASN. This paper proposes a compressive sensing (CS)-based CE solution for full-dimensional (FD) lens-array, where the mmWave channel sparsity is exploited. Specifically, we first propose an approach of pilot training under …
With its privacy preservation and communication efficiency, federated learning (FL) has emerged as a promising learning framework for beyond 5G wireless networks. It is anticipated that future wireless networks will jointly serve both FL and downlink non-FL user groups in the same time-frequency resource. While in the downlink of each FL iteration, both groups jointly receive data from the base station in the same time-frequency resource, the uplink of each FL iteration requires bidirectional communication to support uplink transmission for FL users and downlink transmission for non-FL users. …
We put forward a new algorithmic solution to the massive unsourced random access (URA) problem, by leveraging the rich spatial dimensionality offered by large-scale antenna arrays. This paper makes an observation that spatial signature is key to URA in massive connectivity setups. The proposed scheme relies on a slotted transmission framework but eliminates the need for concatenated coding that was introduced in the context of the coupled compressive sensing (CCS) paradigm. Indeed, all existing works on CCS-based URA rely on an inner/outer tree-based encoder/decoder to stitch the slot-wise rec…
This paper proposes a novel scheme for cell-free massive multiple-input multiple-output (CFmMIMO) networks to support any federated learning (FL) framework. This scheme allows each instead of all the iterations of the FL framework to happen in a large-scale coherence time to guarantee a stable operation of an FL process. To show how to optimize the FL performance using this proposed scheme, we consider an existing FL framework as an example and target FL training time minimization for this framework. An optimization problem is then formulated to jointly optimize the local accuracy, transmit po…
CSI-GPT: Integrating Generative Pre-Trained Transformer with Federated-Tuning to Acquire Downlink Massive MIMO Channels
School of Information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）
Cell-free massive MIMO (CF mMIMO) is a promising next generation wireless architecture to realize federated learning (FL). However, sensitive information of user equipments (UEs) may be exposed to the involved access points or the central processing unit in practice. To guarantee data privacy, effective privacy-preserving mechanisms are defined in this paper. In particular, we demonstrate and characterize the possibility in exploiting the inherent quantization error, caused by low-resolution analog-to-digital converters (ADCs) and digital-to-analog converters (DACs), for privacy-preserving in …
In this paper we consider the problem of sparse signal recovery in Multiple Measurement Vectors (MMVs) case. Recently, ample researches have been conducted to solve this problem and diverse methods are proposed, one of which is deep neural network approach. Here, employing deep neural networks we have provided two new greedy algorithms in order to solve MMV problems. In the first algorithm, we create a stacked vector of measurement matrix columns and a new measurement matrix, which can be assumed as the Kronecker product of the primary compressive sampling matrix and a unitary matrix. Afterwar…
This paper proposes compressed domain signal processing (CSP) multiple input multiple output (MIMO) radar, a MIMO radar approach that achieves substantial sample complexity reduction by exploiting the idea of CSP. CSP MIMO radar involves two levels of data compression followed by target detection at the compressed domain. First, compressive sensing is applied at the receive antennas, followed by a Capon beamformer which is designed to suppress clutter. Exploiting the sparse nature of the beamformer output, a second compression is applied to the filtered data. Target detection is subsequently c…
We present a compressive radar design that combines multitone linear frequency modulated (LFM) waveforms in the transmitter with a classical stretch processor and sub-Nyquist sampling in the receiver. The proposed compressive illumination scheme has fewer random elements resulting in reduced storage and complexity for implementation than previously proposed compressive radar designs based on stochastic waveforms. We analyze this illumination scheme for the task of a joint range-angle of arrival estimation in the multi-input and multi-output (MIMO) radar system. We present recovery guarantees f…
Channel state information (CSI) feedback is critical for frequency division duplex (FDD) massive multi-input multi-output (MIMO) systems. Most conventional algorithms are based on compressive sensing (CS) and are highly dependent on the level of channel sparsity. To address the issue, a recent approach adopts deep learning (DL) to compress CSI into a codeword with low dimensionality, which has shown much better performance than the CS algorithms when feedback link is perfect. In practical scenario, however, there exists various interference and non-linear effect. In this article, we design a D…
