This letter presents a fast reinforcement learning algorithm for anti-jamming communications which chooses previous action with probability $τ$ and applies $ε$-greedy with probability $(1-τ)$. A dynamic threshold based on the average value of previous several actions is designed and probability $τ$ is formulated as a Gaussian-like function to guide the wireless devices. As a concrete example, the proposed algorithm is implemented in a wireless communication system against multiple jammers. Experimental results demonstrate that the proposed algorithm exceeds Q-learing, deep Q-networks (DQN), double DQN (DDQN), and prioritized experience reply based DDQN (PDDQN), in terms of signal-to-interference-plus-noise ratio and convergence rate.
École de technologie supérieure（高等技术学院）；Virginia Tech（弗吉尼亚理工学院暨州立大学）
Conventional anti-jamming methods mainly focus on preventing single jammer attacks with an invariant jamming policy or jamming attacks from multiple jammers with similar jamming policies. These anti-jamming methods are ineffective against a single jammer following several different jamming policies or multiple jammers with distinct policies. Therefore, this paper proposes an anti-jamming method that can adapt its policy to the current jamming attack. Moreover, for the multiple jammers scenario, an anti-jamming method that estimates the future occupied channels using the jammers' occupied chann…
As the applications of deep reinforcement learning (DRL) in wireless communications grow, sensitivity of DRL based wireless communication strategies against adversarial attacks has started to draw increasing attention. In order to address such sensitivity and alleviate the resulting security concerns, we in this paper consider a victim user that performs DRL-based dynamic channel access, and an attacker that executes DRLbased jamming attacks to disrupt the victim. Hence, both the victim and attacker are DRL agents and can interact with each other, retrain their models, and adapt to opponents' …
Intelligent Automation Inc.（智能自动化公司）；U.S. Army C5ISR（美国陆军C5ISR）；University of Minnesota（明尼苏达大学）；The Pennsylvania State University（宾夕法尼亚州立大学）
The problem of quality of service (QoS) and jamming-aware communications is considered in an adversarial wireless network subject to external eavesdropping and jamming attacks. To ensure robust communication against jamming, an interference-aware routing protocol is developed that allows nodes to avoid communication holes created by jamming attacks. Then, a distributed cooperation framework, based on deep reinforcement learning, is proposed that allows nodes to assess network conditions and make deep learning-driven, distributed, and real-time decisions on whether to participate in data commun…
By using smart radio devices, a jammer can dynamically change its jamming policy based on opposing security mechanisms; it can even induce the mobile device to enter a specific communication mode and then launch the jamming policy accordingly. On the other hand, mobile devices can exploit spread spectrum and user mobility to address both jamming and interference. In this paper, a two-dimensional anti-jamming mobile communication scheme is proposed in which a mobile device leaves a heavily jammed/interfered-with frequency or area. It is shown that, by applying reinforcement learning techniques,…
École de technologie supérieure（特克诺高等学院）；Virginia Tech（弗吉尼亚理工大学）
Conventional anti-jamming method mostly rely on frequency hopping to hide or escape from jammer. These approaches are not efficient in terms of bandwidth usage and can also result in a high probability of jamming. Different from existing works, in this paper, a novel anti-jamming strategy is proposed based on the idea of deceiving the jammer into attacking a victim channel while maintaining the communications of legitimate users in safe channels. Since the jammer's channel information is not known to the users, an optimal channel selection scheme and a sub optimal power allocation are proposed…
A Fast Anti-Jamming Cognitive Radar Deployment Algorithm Based on Reinforcement Learning
School of Mathematical Sciences University of Chinese Academy of Sciences（数学科学学院 中国科学院大学）；Information Science Academy China Electronics Technology Group Corporation（信息科学院 中国电子科技集团）
Fast Adaptive Anti-Jamming Channel Access via Deep Q Learning and Coarse-Grained Spectrum Prediction
School of Communications and Information Engineering（通信与信息工程学院）；Nanjing University of Posts and Telecommunications（南京邮电大学）；School of Computer Engineering（计算机工程学院）；Nanjing Institute of Technology（南京理工大学）；National Engineering Research Center of Communications and Networking（通信与网络国家工程研究中心）；College of Communications Engineering（通信工程学院）；Army Engineering University of PLA（中国人民解放军陆军工程大学）
Multi-Agent Deep Reinforcement Learning for Collaborative UAV Relay Networks under Jamming Atatcks
VNU - University of Engineering and Technology (VNU - UET)（越南工程与技术大学）；School of Computer Science and Informatics, University of Liverpool（利兹大学计算机科学与信息学学院）；Interdisciplinary Centre for Security Reliability and Trust (SnT), University of Luxembourg（卢森堡大学安全、可靠性与信任跨学科研究中心）
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
For multi-band wireless ad hoc networks of multiple users, an anti-jamming game between the users and a jammer is studied. In this game, the users (resp. jammer) want to maximize (resp. minimize) the expected rewards of the users taking into account various factors such as communication rate, hopping cost, and jamming loss. We analyze the arms race of the game and derive an optimal frequency hopping policy at each stage of the arms race based on the Markov decision process (MDP). It is analytically shown that the arms race reaches an equilibrium after a few rounds, and a frequency hopping poli…
University of Genoa（热那亚大学）；Queen Mary University of London（伦敦玛丽女王大学）；Italian National Consortium for Telecommunications（意大利国家电信联盟）
This work proposes a novel resource allocation strategy for anti-jamming in Cognitive Radio using Active Inference ($\textit{AIn}$), and a cognitive-UAV is employed as a case study. An Active Generalized Dynamic Bayesian Network (Active-GDBN) is proposed to represent the external environment that jointly encodes the physical signal dynamics and the dynamic interaction between UAV and jammer in the spectrum. We cast the action and planning as a Bayesian inference problem that can be solved by avoiding surprising states (minimizing abnormality) during online learning. Simulation results verify t…
Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks. In this paper, we aim to find collision-free paths for multiple cellular-connected UAVs, while satisfying requirements of connectivity with ground base stations (GBSs) in the presence of a dynamic jammer. We first formulate the problem as a sequential decision making problem in discrete domain, with connectivity, collision avoidance, and kinematic constraints. We, then, propose an offline temporal difference (TD) learning algorithm with online signal-to-interference-plus-noise ratio (SINR) mapping to solv…
Deep reinforcement learning (DRL) has recently been used to perform efficient resource allocation in wireless communications. In this paper, the vulnerabilities of such DRL agents to adversarial attacks is studied. In particular, we consider multiple DRL agents that perform both dynamic channel access and power control in wireless interference channels. For these victim DRL agents, we design a jammer, which is also a DRL agent. We propose an adversarial jamming attack scheme that utilizes a listening phase and significantly degrades the users' sum rate. Subsequently, we develop an ensemble pol…
Anti-Jamming Sensing with Distributed Reconfigurable Intelligent Metasurface Antennas
School of Information and Communications Engineering, Xi’an Jiaotong University（信息与通信工程学院，西安交通大学）；Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University（教育部智能网络与网络安全重点实验室，西安交通大学）
Can an intelligent jammer learn and adapt to unknown environments in an electronic warfare-type scenario? In this paper, we answer this question in the positive, by developing a cognitive jammer that adaptively and optimally disrupts the communication between a victim transmitter-receiver pair. We formalize the problem using a novel multi-armed bandit framework where the jammer can choose various physical layer parameters such as the signaling scheme, power level and the on-off/pulsing duration in an attempt to obtain power efficient jamming strategies. We first present novel online learning a…
This paper investigates the anti-jamming performance of a cognitive radar under a partially observable Markov decision process (POMDP) model. First, we obtain an explicit expression for uncertainty of jammer dynamics, which paves the way for illuminating the performance metric of probability of being jammed for the radar beyond a conventional signal-to-noise ratio ($\mathsf{SNR}$) based analysis. Considering two frequency hopping strategies developed in the framework of reinforcement learning (RL), this performance metric is analyzed with deep Q-network (DQN) and long short term memory (LSTM) …
Cellular systems are vulnerable to jamming attacks, especially smart jammers that choose their jamming policies such as the jamming channel frequencies and power based on the ongoing communication policies and network states. In this article, we present an unmanned aerial vehicle (UAV) aided cellular communication framework against jamming. In this scheme, UAVs use reinforcement learning methods to choose the relay policy for mobile users in cellular systems, if the serving base station is heavily jammed. More specifically, we propose a deep reinforcement learning based UAV relay scheme to hel…
PCA-Featured Transformer for Jamming Detection in 5G UAV Networks
UC3M - Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；ISCTE – Instituto Universitário de Lisboa（里斯本大学学院）；Tyndall National Institute（廷德尔国家研究所）；IT – Instituto de Telecomunicações（电信研究所）
In this letter, we investigate the anti-jamming defense problem in multi-user scenarios, where the coordination among users is taken into consideration. The Markov game framework is employed to model and analyze the anti-jamming defense problem, and a collaborative multi-agent anti-jamming algorithm (CMAA) is proposed to obtain the optimal anti-jamming strategy. In sweep jamming scenarios, on the one hand, the proposed CMAA can tackle the external malicious jamming. On the other hand, it can effectively cope with the mutual interference among users. Simulation results show that the proposed CM…
Malicious jamming launched by smart jammer, which attacks legitimate transmissions has been regarded as one of the critical security challenges in wireless communications. Thus, this paper exploits intelligent reflecting surface (IRS) to enhance anti-jamming communication performance and mitigate jamming interference by adjusting the surface reflecting elements at the IRS. Aiming to enhance the communication performance against smart jammer, an optimization problem for jointly optimizing power allocation at the base station (BS) and reflecting beamforming at the IRS is formulated. As the jammi…
This paper presents a new deep reinforcement learning (DRL)-based approach to the trajectory planning and jamming rejection of an unmanned aerial vehicle (UAV) for the Internet-of-Things (IoT) applications. Jamming can prevent timely delivery of sensing data and reception of operation instructions. With the assistance of a reconfigurable intelligent surface (RIS), we propose to augment the radio environment, suppress jamming signals, and enhance the desired signals. The UAV is designed to learn its trajectory and the RIS configuration based solely on changes in its received data rate, using th…
Malicious jamming launched by smart jammers can attack legitimate transmissions, which has been regarded as one of the critical security challenges in wireless communications. With this focus, this paper considers the use of an intelligent reflecting surface (IRS) to enhance anti-jamming communication performance and mitigate jamming interference by adjusting the surface reflecting elements at the IRS. Aiming to enhance the communication performance against a smart jammer, an optimization problem for jointly optimizing power allocation at the base station (BS), and reflecting beamforming at th…
Georgia Institute of Technology（佐治亚理工学院）；IBM Research（IBM研究院）；Preferred Network America（美国首选网络公司）；Columbia University（哥伦比亚大学）
Recent deep neural networks based techniques, especially those equipped with the ability of self-adaptation in the system level such as deep reinforcement learning (DRL), are shown to possess many advantages of optimizing robot learning systems (e.g., autonomous navigation and continuous robot arm control.) However, the learning-based systems and the associated models may be threatened by the risks of intentionally adaptive (e.g., noisy sensor confusion) and adversarial perturbations from real-world scenarios. In this paper, we introduce timing-based adversarial strategies against a DRL-based …
Frequency hopping (FH) is an effective anti-jamming technology in cognitive radio networks (CRNs). However, it is difficult to significantly increase the anti-jamming results because of the growing crowded spectrum in wireless communications. Orbital momentum angular (OAM) provides a new mode dimension for anti-jamming without consuming extra power and frequency resources in CRNs. In this paper, we propose the mode-frequency hopping (MFH) scheme, which jointly uses the mode hopping and the traditional FH scheme for anti-jamming to significantly increase the capacity of secondary users (SUs). W…
We consider the problem of designing waveforms for mitigating single tone jamming (STJ) signals with an estimated jamming frequency in time-hopping spread spectrum (TH SS) systems. The proposed design of waveforms optimizes the anti-jamming (AJ) performance of TH SS systems by minimizing the correlation between the template and STJ signals, in which the problem of waveform optimization is simplified by employing a finite number of rectangular pulses. The simplification eventually makes the design of waveforms be converted into a problem of finding eigenvalues and eigenvectors of a matrix. Simu…
Stable Online and Offline Reinforcement Learning for Antibody CDRH3 Design
University of Freiburg（弗赖堡大学）；Collaborative Research Institute Intelligent Oncology (CRIION)（智能肿瘤学合作研究所（CRIION））；University Medical Center Freiburg（弗赖堡大学医学中心）；BrainLinks-BrainTools（脑连接-脑工具研究中心）
Frequency Hopping Synchronization by Reinforcement Learning for Satellite Communication System
Korea Aerospace Research Institute (KARI)（韩国航空航天研究院（KARI））；Hanbat National University（韩bat国立大学）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Providing quality communications under adversarial electronic attacks, e.g., broadband jamming attacks, is a challenging task. Unlike state-of-the-art approaches which treat jamming signals as destructive interference, this paper presents a novel active anti-jamming (AAJ) scheme for a jammed channel to enhance the communication quality between a transmitter node (TN) and receiver node (RN), where the TN actively exploits the jamming signal as a carrier to send messages. Specifically, the TN is equipped with a programmable-gain amplifier, which is capable of re-modulating the jamming signals fo…
University of Geneva（日内瓦大学）；Google Research（谷歌研究院）；Univ. Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Univ. Lille（里尔大学）
Offline Reinforcement Learning (RL) aims at learning an optimal control from a fixed dataset, without interactions with the system. An agent in this setting should avoid selecting actions whose consequences cannot be predicted from the data. This is the converse of exploration in RL, which favors such actions. We thus take inspiration from the literature on bonus-based exploration to design a new offline RL agent. The core idea is to subtract a prediction-based exploration bonus from the reward, instead of adding it for exploration. This allows the policy to stay close to the support of the da…
The navigation of robots in dynamic urban environments, requires elaborated anticipative strategies for the robot to avoid collisions with dynamic objects, like bicycles or pedestrians, and to be human aware. We have developed and analyzed three anticipative strategies in motion planning taking into account the future motion of the mobile objects that can move up to 18 km/h. First, we have used our hybrid policy resulting from a Deep Deterministic Policy Gradient (DDPG) training and the Social Force Model (SFM), and we have tested it in simulation in four complex map scenarios with many pedest…
In millimeter-wave (MMW) multiple-input multiple-output (MIMO) communications, users and their corresponding base station (BS) have to align their beam during both initial access and data transmissions to compensate for the high propagation loss. The beam alignment (BA) procedure specified for 5th Generation (5G) New Radio (NR) has been designed to be fast and precise in the presence of non-malicious interference and noise. A smart jammer might exploit this weakness and may launch an attack during the BA phase in order to degrade the accuracy of beam selection and, thus, adversely impacting th…
