Random access (RA) schemes are a topic of high interest in machine-type communication (MTC). In RA protocols, backoff techniques such as exponential backoff (EB) are used to stabilize the system to avoid low throughput and excessive delays. However, these backoff techniques show varying performance for different underlying assumptions and analytical models. Therefore, finding a better transmission policy for slotted ALOHA RA is still a challenge. In this paper, we show the potential of deep reinforcement learning (DRL) for RA. We learn a transmission policy that balances between throughput and fairness. The proposed algorithm learns transmission probabilities using previous action and binary feedback signal, and it is adaptive to different traffic arrival rates. Moreover, we propose average age of packet (AoP) as a metric to measure fairness among users. Our results show that the proposed policy outperforms the baseline EB transmission schemes in terms of throughput and fairness.
This paper studies multi-agent deep reinforcement learning (MADRL) based resource allocation methods for multi-cell wireless powered communication networks (WPCNs) where multiple hybrid access points (H-APs) wirelessly charge energy-limited users to collect data from them. We design a distributed reinforcement learning strategy where H-APs individually determine time and power allocation variables. Unlike traditional centralized optimization algorithms which require global information collected at a central unit, the proposed MADRL technique models an H-AP as an agent producing its action base…
We consider an ambient backscatter communication (AmBC) system aided by an intelligent reflecting surface (IRS). The optimization of the IRS to assist AmBC is extremely difficult when there is no prior channel knowledge, for which no design solutions are currently available. We utilize a deep reinforcement learning-based framework to jointly optimize the IRS and reader beamforming, with no knowledge of the channels or ambient signal. We show that the proposed framework can facilitate effective AmBC communication with a detection performance comparable to several benchmarks under full channel k…
Motivated by the proliferation of real-time applications in multimedia communication systems, tactile Internet, and cyber-physical systems, supporting delay-constrained traffic becomes critical for such systems. In delay-constrained traffic, each packet has a hard deadline; when it is not delivered before its deadline is up, it becomes useless and will be removed from the system. In this work, we focus on designing random access schemes for delay-constrained wireless communications. We first investigate three ALOHA-based schemes and prove that the system timely throughput of all three schemes …
Nanyang Technological University（南洋理工大学）；University of Technology Sydney（悉尼科技大学）；Sungkyunkwan University (SKKU)（成均馆大学）
This letter introduces a novel framework to optimize the power allocation for users in a Rate Splitting Multiple Access (RSMA) network. In the network, messages intended for users are split into different parts that are a single common part and respective private parts. This mechanism enables RSMA to flexibly manage interference and thus enhance energy and spectral efficiency. Although possessing outstanding advantages, optimizing power allocation in RSMA is very challenging under the uncertainty of the communication channel and the transmitter has limited knowledge of the channel information.…
We consider an Intelligent Reflecting Surface (IRS)-aided multiple-input single-output (MISO) system for downlink transmission. We compare the performance of Deep Reinforcement Learning (DRL) and conventional optimization methods in finding optimal phase shifts of the IRS elements to maximize the user signal-to-noise (SNR) ratio. Furthermore, we evaluate the robustness of these methods to channel impairments and changes in the system. We demonstrate numerically that DRL solutions show more robustness to noisy channels and user mobility.
In this paper, a novel communication framework that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed to enhance multi-user downlink transmissions over millimeter wave (mmWave) frequencies. In order to maximize the downlink sum-rate, the optimal precoding matrix (at the base station) and reflection coefficient (at the IR) are jointly derived. Next, to address the uncertainty of mmWave channels and maintain line-of-sight links in a real-time manner, a distributional reinforcement learning approach, based on quantile regression optimization, is proposed to lear…
In this paper, the placement strategy design of coded caching in fog-radio access networks (F-RANs) is investigated. By considering time-variant content popularity, federated deep reinforcement learning is exploited to learn the placement strategy for our coded caching scheme. Initially, the placement problem is modeled as a Markov decision process (MDP) to capture the popularity variations and minimize the long-term content access delay. The reformulated sequential decision problem is solved by dueling double deep Q-learning (dueling DDQL). Then, federated learning is applied to learn the rel…
Deep reinforcement learning (DRL) has recently been used to perform efficient resource allocation in wireless communications. In this paper, the vulnerabilities of such DRL agents to adversarial attacks is studied. In particular, we consider multiple DRL agents that perform both dynamic channel access and power control in wireless interference channels. For these victim DRL agents, we design a jammer, which is also a DRL agent. We propose an adversarial jamming attack scheme that utilizes a listening phase and significantly degrades the users' sum rate. Subsequently, we develop an ensemble pol…
Most of the current anti-jamming algorithms for wireless communications only consider how to avoid jamming attacks, but ignore that the communication waveform or frequency action may be obtained by the jammers. Although existing anti-jamming methods can guarantee temporary communication effects, the long-term performance of these anti-jamming methods may be depressed when intelligent jammers are capable of learning from historical communication activities. Aiming at this issue, a hidden anti-jamming method based on the idea of reducing the jammer's sense probability is proposed. Firstly, the s…
Grant-free random access (RA) techniques are suitable for machine-type communication (MTC) networks but they need to be adaptive to the MTC traffic, which is different from the human-type communication. Conventional RA protocols such as exponential backoff (EB) schemes for slotted-ALOHA suffer from a high number of collisions and they are not directly applicable to the MTC traffic models. In this work, we propose to use multi-agent deep Q-network (DQN) with parameter sharing to find a single policy applied to all machine-type devices (MTDs) in the network to resolve collisions. Moreover, we co…
Rate-Splitting Multiple Access (RSMA) is an emerging flexible and powerful multiple access for downlink multiantenna networks. In this paper, we introduce the concept of RSMA into short-packet downlink communications. We design optimal linear precoders that maximize the sum rate with Finite Blocklength (FBL) constraints. The relations between the sum rate and blocklength of RSMA are investigated for a wide range of network loads and user deployments. Numerical results demonstrate that RSMA can achieve the same transmission rate as Non-Orthogonal Multiple Access (NOMA) and Space Division Multip…
This paper studies the allocation of shared resources between vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) links in vehicle-to-everything (V2X) communications. In existing algorithms, dynamic vehicular environments and quantization of continuous power become the bottlenecks for providing an effective and timely resource allocation policy. In this paper, we develop two algorithms to deal with these difficulties. First, we propose a deep reinforcement learning (DRL)-based resource allocation algorithm to improve the performance of both V2I and V2V links. Specifically, the algorit…
We propose a novel formulation of the "effectiveness problem" in communications, put forth by Shannon and Weaver in their seminal work [2], by considering multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation in a multi-agent reinforcement learning (MARL) framework. Specifically, we consider a multi-agent partially observable Markov decision process (MA-POMDP), in which the agents, in addition to interacting with the environment can also communicate with each other over a noisy communication channel. The noisy communication channel is consid…
Xiamen University（厦门大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Massive random access of devices in the emerging Open Radio Access Network (O-RAN) brings great challenge to the access control and management. Exploiting the bursting nature of the access requests, sparse active user detection (SAUD) is an efficient enabler towards efficient access management, but the sparsity might be deteriorated in case of uncoordinated massive access requests. To dynamically preserve the sparsity of access requests, a reinforcement-learning (RL)-assisted scheme of closed-loop access control utilizing the access class barring technique is proposed, where the RL policy is d…
University of Colorado Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）
Communication at terahertz (THz) frequency bands is a promising solution for achieving extremely high data rates in next-generation wireless networks. While the THz communication is conventionally envisioned for short-range wireless applications due to the high atmospheric absorption at THz frequencies, multi-hop directional transmissions can be enabled to extend the communication range. However, to realize multi-hop THz communications, conventional beam training schemes, such as exhaustive search or hierarchical methods with a fixed number of training levels, can lead to a very large time ove…
Graph Attention-based Reinforcement Learning for Trajectory Design and Resource Assignment in Multi-UAV Assisted Communication
Hainan University（海南大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Shanghai Jiao Tong University（上海交通大学）；University of Naples Federico II（那不勒斯费德里科二世大学）；Nanyang Technological University（南洋理工大学）
