Vision-aided wireless communication is motivated by the recent advances in deep learning and computer vision as well as the increasing dependence on line-of-sight links in millimeter wave (mmWave) and terahertz systems. By leveraging vision, this new research direction enables an interesting set of new capabilities such as vision-aided mmWave beam and blockage prediction, proactive hand-off, and resource allocation among others. These capabilities have the potential of reliably supporting highly-mobile applications such as vehicular/drone communications and wireless virtual/augmented reality in mmWave and terahertz systems. Investigating these interesting applications, however, requires the development of special dataset and machine learning tasks. Based on the Vision-Wireless (ViWi) dataset generation framework [1], this paper develops an advanced and realistic scenario/dataset that features multiple base stations, mobile users, and rich dynamics. Enabled by this dataset, the paper defines the vision-wireless mmWave beam tracking task (ViWi-BT) and proposes a baseline solution that can provide an initial benchmark for the future ViWi-BT algorithms.
The growing role that artificial intelligence and specifically machine learning is playing in shaping the future of wireless communications has opened up many new and intriguing research directions. This paper motivates the research in the novel direction of \textit{vision-aided wireless communications}, which aims at leveraging visual sensory information in tackling wireless communication problems. Like any new research direction driven by machine learning, obtaining a development dataset poses the first and most important challenge to vision-aided wireless communications. This paper addresse…
Location information offered by external positioning systems, e.g., satellite navigation, can be used as prior information in the process of beam alignment and channel parameter estimation for reconfigurable intelligent surface (RIS)-aided millimeter wave (mmWave) multiple-input multiple-output networks. Benefiting from the availability of such prior information, albeit imperfect, the beam alignment and channel parameter estimation processes can be significantly accelerated with less candidate beams explored at all the terminals. We propose a practical channel parameter estimation method via a…
Compressed Sensing (CS) based channel estimation techniques have recently emerged as an effective way to acquire the channel of millimeter-wave (mmWave) systems with a small number of measurements. These techniques, however, are based on prior knowledge of transmit and receive array manifolds, and assume perfect antenna arrays at both the transmitter and the receiver. In the presence of antenna imperfections, the geometry and response of the arrays are modified. This distorts the CS measurement matrix and results in channel estimation errors. This paper studies the effects of both transmit and…
A reconfigurable intelligent surface (RIS) can shape the radio propagation by passively changing the directions of impinging electromagnetic waves. The optimal control of the RIS requires perfect channel state information (CSI) of all the links connecting the base station (BS) and the mobile station (MS) via the RIS. Thereby the channel (parameter) estimation at the BS/MS and the related message feedback mechanism are needed. In this paper, we adopt a two-stage channel estimation scheme for the RIS-aided millimeter wave (mmWave) MIMO channels using an iterative reweighted method to sequentiall…
Intelligent reflecting surface (IRS) has been considered as a promising technology to alleviate the blockage effect and enhance coverage in millimeter wave (mmWave) communication. To explore the impact of IRS on the performance of mmWave communication, we investigate a multi-IRS assisted mmWave communication network and formulate a sum rate maximization problem by jointly optimizing the active and passive beamforming and the set of IRSs for assistance. The optimization problem is intractable due to the lack of convexity of the objective function and the binary nature of the IRS selection varia…
This paper presents an inertial sensor aided technique for beam alignment and tracking in massive multiple-input multiple-output (MIMO) vehicle-to-vehicle (V2V) communications based on millimeter waves (mmWave). Since directional communications in vehicular scenarios are severely hindered by beam pointing issues, a beam alignment procedure has to be periodically carried out to guarantee the communication reliability. When dealing with massive MIMO links, the beam sweeping approach is known to be time consuming and often unfeasible due to latency constraints. To speed up the process, we propose…
As communication systems that employ millimeter wave (mmWave) frequency bands must use large antenna arrays to overcome the severe propagation loss of mmWave signals, hybrid beamforming has been considered as an integral component of mmWave communications. Recently, intelligent reflecting surface (IRS) has been proposed as an innovative technology that can significantly improve the performance of mmWave communication systems through the use of low-cost passive reflecting elements. In this paper, we study IRS-aided mmWave multiple-input multiple-output (MIMO) systems with hybrid beamforming arc…
Millimeter wave (mmWave) massive multiple-input multiple-output (massive MIMO) is one of the most promising technologies for the fifth generation and beyond wireless communication system. However, a large number of antennas incur high power consumption and hardware costs, and high-frequency communications place a heavy burden on the analog-to-digital converters (ADCs) at the base station (BS). Furthermore, it is too costly to equipping each antenna with a high-precision ADC in a large antenna array system. It is promising to adopt low-resolution ADCs to address this problem. In this paper, we …
This paper proposes a reconfigurable intelligent surface (RIS)-aided and angular-based hybrid beamforming (AB-HBF) technique for the millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. The proposed RIS-AB-HBF architecture consists of three stages: (i) RF beamformer, (ii) baseband (BB) precoder/combiner, and (iii) RIS phase shift design. First, in order to reduce the number of RF chains and the channel estimation overhead, RF beamformers are designed based on the 3D geometry-based mmWave channel model using slow time-varying angular parameters of the channel. Second,…
A fundamental challenge in millimeter-wave (mmWave) communication is the susceptibility to blocking objects. One way to alleviate this problem is the use of reconfigurable intelligent surfaces (RIS). Nevertheless, due to the large number of passive reflecting elements on RIS, channel estimation turns out to be a challenging task. In this paper, we address the channel estimation for RIS-aided mmWave communication systems based on a localization method. The proposed idea consists of exploiting the sparsity of the mmWave channel and the topology of the RIS. In particular, we first propose the con…
This paper proposes a novel reconfigurable intelligent surface (RIS) architecture which enables channel estimation of RIS-assisted millimeter wave (mmWave) systems. More specifically, two channel estimation methods, namely, direct calculation (DC) and deep learning (DL) methods, are proposed to skillfully convert the overall channel estimation into two tasks: the channel estimation and the angle parameter estimation of a small number of active elements. In particular, the direct calculation method calculates the angle parameters directly through the channel estimates of adjacent active element…
In millimeter wave (mmWave) systems, it is challenging to ensure the reliable connectivity of communications due to its sensitivity to the presence of blockages. In order to improve the robustness of the mmWave system under the presence of the random blockages, multiple reconfigurable intelligent surfaces (RISs) are deployed to enhance the spatial diversity gain, and robust beamforming is then designed based on a stochastic optimization for minimizing the maximum outage probability among multiple users to ensure the fairness. Under the stochastic optimization framework, we adopt the stochastic…
Millimeter wave (mmWave) full-duplex (FD) is a promising technique for improving capacity by maximizing the utilization of both time and the rich mmWave frequency resources. Still, it has restrictions due to FD self-interference (SI) and mmWave's limited coverage. Therefore, this study dives into FD mmWave MIMO with the assistance of reconfigurable intelligent surfaces (RIS) for capacity improvement. First, we demonstrate the angular-domain reciprocity of FD antenna arrays under the far-field planar wavefront assumption. Accordingly, a strategy for joint downlink-uplink (DL-UL) channel estimat…
In this letter, we investigate the robust and secure beamforming (RSBF) in an intelligent reflecting surface (IRS) aided millimeter wave (mmWave) multiple input single output (MISO) system, where multiple single antenna eavesdroppers (Eves) are arbitrarily distributed nearby the legitimate receiver. Considering the channel state information (CSI) of Eves' channels is imperfectly known at the legitimate transmitter, the RSBF design problems to maximize the worst case of achievable secrecy rate (ASR) are formulated under the total transmission power and unit-modulus constraints. Since the proble…
A reconfigurable intelligent surface (RIS) can shape the radio propagation environment by virtue of changing the impinging electromagnetic waves towards any desired directions, thus, breaking the general Snell's reflection law. However, the optimal control of the RIS requires perfect channel state information (CSI) of the individual channels that link the base station (BS) and the mobile station (MS) to each other via the RIS. Thereby super-resolution channel (parameter) estimation needs to be efficiently conducted at the BS or MS with CSI feedback to the RIS controller. In this paper, we adop…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
Hybrid beamforming is widely recognized as an important technique for millimeter wave (mmWave) multiple input multiple output (MIMO) systems. Generalized spatial modulation (GSM) is further introduced to improve the spectrum efficiency. However, most of the existing works on beamforming assume the perfect channel state information (CSI), which is unrealistic in practical systems. In this paper, joint optimization of downlink pilot training, channel estimation, CSI feedback, and hybrid beamforming is considered in GSM aided frequency division duplexing (FDD) mmWave MIMO systems. With the help o…
This paper presents the first machine learning based real-world demonstration for radar-aided beam prediction in a practical vehicular communication scenario. Leveraging radar sensory data at the communication terminals provides important awareness about the transmitter/receiver locations and the surrounding environment. This awareness could be utilized to reduce or even eliminate the beam training overhead in millimeter wave (mmWave) and sub-terahertz (THz) MIMO communication systems, which enables a wide range of highly-mobile low-latency applications. In this paper, we develop deep learning…
Line-of-sight link blockages represent a key challenge for the reliability and latency of millimeter wave (mmWave) and terahertz (THz) communication networks. This paper proposes to leverage LiDAR sensory data to provide awareness about the communication environment and proactively predict dynamic link blockages before they happen. This allows the network to make proactive decisions for hand-off/beam switching which enhances its reliability and latency. We formulate the LiDAR-aided blockage prediction problem and present the first real-world demonstration for LiDAR-aided blockage prediction in…
Millimeter wave (mmWave) communication is expected to play an important role in next generation cellular networks, aiming to cope with the bandwidth shortage affecting conventional wireless carriers. Using side-information has been proposed as a potential approach to accelerate beam selection in mmWave massive MIMO (m-MIMO) communications. However, in practice, such information is not error-free, leading to performance degradation. In the multi-user case, a wrong beam choice might result in irreducible inter-user interference at the base station (BS) side. In this paper, we consider location-a…
Reconfigurable intelligent surface (RIS) is a promising technology for future millimeter-wave (mmWave) communication systems. However, its potential benefits of adopting RIS for high-precision positioning in mmWave systems are still less understood. In this paper, we study a multiple-RIS-aided mmWave positioning system and derive the Cram$\rm{\acute{e}}$r-Rao error bound. Based on the derived bound, we optimize the phase shift of the RISs by the particle swarm optimization (PSO) algorithm. Numerical results have demonstrated the advantages of using multiple RISs in enhancing the positioning ac…
Hybrid analog-digital (A/D) transceivers designed for millimeter wave (mmWave) systems have received substantial research attention, as a benefit of their lower cost and modest energy consumption compared to their fully-digital counterparts. We further improve their performance by conceiving a Tomlinson-Harashima precoding (THP) based nonlinear joint design for the downlink of multiuser multiple-input multiple-output (MIMO) mmWave systems. Our optimization criterion is that of minimizing the mean square error (MSE) of the system under channel uncertainties subject both to realistic transmit po…
This paper investigates a novel research direction that leverages vision to help overcome the critical wireless communication challenges. In particular, this paper considers millimeter wave (mmWave) communication systems, which are principal components of 5G and beyond. These systems face two important challenges: (i) the large training overhead associated with selecting the optimal beam and (ii) the reliability challenge due to the high sensitivity to link blockages. Interestingly, most of the devices that employ mmWave arrays will likely also use cameras, such as 5G phones, self-driving vehi…
The problem of beam alignment and tracking in high mobility scenarios such as high-speed railway (HSR) becomes extremely challenging, since large overhead cost and significant time delay are introduced for fast time-varying channel estimation. To tackle this challenge, we propose a learning-aided beam prediction scheme for HSR networks, which predicts the beam directions and the channel amplitudes within a period of future time with fine time granularity, using a group of observations. Concretely, we transform the problem of high-dimensional beam prediction into a two-stage task, i.e., a low-d…
GPS-Aided Deep Learning for Beam Prediction and Tracking in UAV mmWave Communication
Department of Intelligent Mechatronics Engineering, and Department of Convergence Engineering for Intelligent Drone, Sejong University（智能机电工程系，智能无人机融合工程系，世宗大学）
Current scene depth estimation approaches mainly rely on optical sensing, which carries privacy concerns and suffers from estimation ambiguity for distant, shiny, and transparent surfaces/objects. Reconfigurable intelligent surfaces (RISs) provide a path for employing a massive number of antennas using low-cost and energy-efficient architectures. This has the potential for realizing RIS-aided wireless sensing with high spatial resolution. In this paper, we propose to employ RIS-aided wireless sensing systems for scene depth estimation. We develop a comprehensive framework for building accurate…
EURECOM（欧洲通信系统研究院）；University of Surrey（萨里大学）；Imperial College London（帝国理工学院）
Efficient millimeter wave (mmWave) beam selection in vehicle-to-infrastructure (V2I) communication is a crucial yet challenging task due to the narrow mmWave beamwidth and high user mobility. To reduce the search overhead of iterative beam discovery procedures, contextual information from light detection and ranging (LIDAR) sensors mounted on vehicles has been leveraged by data-driven methods to produce useful side information. In this paper, we propose a lightweight neural network (NN) architecture along with the corresponding LIDAR preprocessing, which significantly outperforms previous work…
This paper presents the first large-scale real-world evaluation for using LiDAR data to guide the mmWave beam prediction task. A machine learning (ML) model that leverages the LiDAR sensory data to predict the current and future beams was developed. Based on the large-scale real-world dataset, DeepSense 6G, this model was evaluated in a vehicle-to-infrastructure communication scenario with highly-mobile vehicles. The experimental results show that the developed LiDAR-aided beam prediction and tracking model can predict the optimal beam in $95\%$ of the cases and with more than $90\%$ reduction…
Reconfigurable intelligent surface (RIS) is an attractive technology to improve the transmission rate of millimetre-wave (mmWave) communication systems. The previous {research} on RIS technology mainly focused on improving the transmission rate and security rate of the mmWave communication systems. Since the emergence of RIS technology creates the conditions for generating an intelligent radio environment, it also has potential advantages on improving the localization accuracy of the mmWave communication systems. Deployed on walls and objects, RISs are capable of significantly improving commun…
In this letter, we focus on the hybrid precoding (HP) design for the reconfigurable intelligent surface (RIS) aided multi-user (MU) millimeter wave (mmWave) communication systems. Specifically, we aim to minimize the mean-squared-error (MSE) between the received symbols and the transmitted symbols by jointly optimizing the analog-digital HP at the base-station (BS) and the phase shifts (PSs) at the RIS, where the non-convex element-wise constant-modulus constraints for the analog precoding and the PSs are tackled by resorting to the gradient-projection (GP) method. We analytically prove the co…
RIS-aided millimeter wave wireless systems benefit from robustness to blockage and enhanced coverage. In this paper, we study the ability of RIS to also provide enhanced localization capabilities as a by-product of communication. We consider sparse reconstruction algorithms to obtain high resolution channel estimates that are mapped to position information. In RIS-aided mmWave systems, the complexity of sparse recovery becomes a bottleneck, given the large number of elements of the RIS and the large communication arrays. We propose to exploit a multidimensional orthogonal matching pursuit stra…
In this work, we explore the potential benefits of deploying unmanned aerial vehicles (UAVs) as aerial base stations (ABSs) with sub-6GHz band and small cells terrestrial base stations (TBSs) with millimeter wave (mmWave) band in a hybrid heterogeneous networks (HetNets). A flexible non-orthogonal multiple access (NOMA) based user association policy is proposed. By using the tools from stochastic geometry, new analytical expressions for association probability, coverage probability and spectrum efficiency are derived for characterizing the performance of UAV-aided HetNets under the realistic A…
In this paper, we propose a novel two-stage based uplink channel estimation strategy with reduced pilot overhead and error propagation for a reconfigurable intelligent surface (RIS)-aided multi-user (MU) millimeter wave (mmWave) system. Specifically, in Stage I, with the carefully designed RIS phase shift matrix and introduced matching matrices, all users jointly estimate the correlation factors between different paths of the common RIS-base station (BS) channel, which achieves significant multi-user diversity gain. Then, the inherent scaling ambiguity and angle ambiguity of the mmWave cascade…
University of Oulu（奥卢大学）；Chalmers University of Technology（查尔姆斯理工大学）；Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；CentraleSupélec（中央理工-高等电力学院）
Inspired by the remarkable learning and prediction performance of deep neural networks (DNNs), we apply one special type of DNN framework, known as model-driven deep unfolding neural network, to reconfigurable intelligent surface (RIS)-aided millimeter wave (mmWave) single-input multiple-output (SIMO) systems. We focus on uplink cascaded channel estimation, where known and fixed base station combining and RIS phase control matrices are considered for collecting observations. To boost the estimation performance and reduce the training overhead, the inherent channel sparsity of mmWave channels i…
In this letter, secure beamforming in a multiple intelligent reflecting surfaces (IRSs)-aided millimeter-wave (mmWave) system is investigated. In this system, the secrecy rate is maximized by controlling the on-off status of each IRS as well as optimizing the phase shift matrix of the IRSs. This problem is posed as a joint optimization problem of transmit beamforming and IRS control, whose goal is to maximize the secrecy rate under the total transmission power and unit-modulus constraints. The problem is difficult to solve optimally due to the nonconvexity of constraint conditions and coupled …
In this paper, an reconfigurable intelligent surface (RIS)-aided millimeter wave (mmWave) non-orthogonal multiple access (NOMA) system is considered. In particular, we consider an RIS-aided mmWave-NOMA downlink system with a hybrid beamforming structure. To maximize the achievable sum-rate under a minimum rate constraint for the users and a minimum transmit power constraint, a joint RIS phase shifts, hybrid beamforming, and power allocation problem is formulated. To solve this non-convex optimization problem, we develop an alternating optimization algorithm. Specifically, first, the non-convex…
In this paper, we propose an environment semantics aided wireless communication framework to reduce the transmission latency and improve the transmission reliability, where semantic information is extracted from environment image data, selectively encoded based on its task-relevance, and then fused to make decisions for channel related tasks. As a case study, we develop an environment semantics aided network architecture for mmWave communication systems, which is composed of a semantic feature extraction network, a feature selection algorithm, a task-oriented encoder, and a decision network. W…
University of Electronic Science and Technology of China（电子科技大学）；Yibin Institute of UESTC（电子科技大学宜宾研究院）；Zhejiang University（浙江大学）；Singapore University of Technology and Design（新加坡科技设计大学）
Reconfigurable intelligent surface (RIS) has recently gained popularity as a promising solution for improving the signal transmission quality of wireless communications with less hardware cost and energy consumption. This letter offers a novel deep reinforcement learning (DRL) algorithm based on a location-aware imitation environment for the joint beamforming design in an RIS-aided mmWave multiple-input multiple-output system. Specifically, we design a neural network to imitate the transmission environment based on the geometric relationship between the user's location and the mmWave channel. …
MmWave multiple-input single-output (MISO) systems using a single-antenna receiver are regarded as promising solution for the near future, before the full-fledged 5G MIMO will be widespread. However, for MISO systems synchronization cannot be performed jointly with user localization unless two-way transmissions are used. In this paper we show that thanks to the use of a reconfigurable intelligent surface (RIS), joint localization and synchronization is possible with only downlink MISO transmissions. The direct maximum likelihood (ML) estimator for the position and clock offset is derived. To o…
Reconfigurable intelligent surface (RIS) is a promising technique for millimeter wave (mmWave) positioning systems. In this paper, we consider multiple mobile users (MUs) positioning problem in the multiple-input multiple-output (MIMO) time-division duplex (TDD) mmWave systems aided by the RIS. We derive the expression for the space-time channel response vector (STCRV) as a novel type of fingerprint. The STCRV consists of the multipath channel characteristics, e.g., time delay and angle of arrival (AOA), which is related to the position of the MU. By using the STCRV as input, we propose a nove…
The concept of reconfigurable intelligent surface (RIS) has been proposed to change the propagation of electromagnetic waves, e.g., reflection, diffraction, and refraction. To accomplish this goal, the phase values of the discrete RIS units need to be optimized. In this paper, we consider RIS-aided millimeter-wave (mmWave) multiple-input multiple-output (MIMO) systems for both accurate positioning and high data-rate transmission. We propose an adaptive phase shifter design based on hierarchical codebooks and feedback from the mobile station (MS). The benefit of the scheme lies in that the RIS …
Millimeter wave (mmWave) and sub-terahertz communication systems rely mainly on line-of-sight (LOS) links between the transmitters and receivers. The sensitivity of these high-frequency LOS links to blockages, however, challenges the reliability and latency requirements of these communication networks. In this paper, we propose to utilize radar sensors to provide sensing information about the surrounding environment and moving objects, and leverage this information to proactively predict future link blockages before they happen. This is motivated by the low cost of the radar sensors, their abi…
