This paper studies the path design problem for cellular-connected unmanned aerial vehicle (UAV), which aims to minimize its mission completion time while maintaining good connectivity with the cellular network. We first argue that the conventional path design approach via formulating and solving optimization problems faces several practical challenges, and then propose a new reinforcement learning-based UAV path design algorithm by applying \emph{temporal-difference} method to directly learn the \emph{state-value function} of the corresponding Markov Decision Process. The proposed algorithm is further extended by using linear function approximation with tile coding to deal with large state space. The proposed algorithms only require the raw measured or simulation-generated signal strength as the input and are suitable for both online and offline implementations. Numerical results show that the proposed path designs can successfully avoid the coverage holes of cellular networks even in the complex urban environment.
King’s College London（伦敦国王学院）；Ericsson Inc.（爱立信公司）
Cellular-connected unmanned aerial vehicle (UAV) swarm is a promising solution for diverse applications, including cargo delivery and traffic control. However, it is still challenging to communicate with and control the UAV swarm with high reliability, low latency, and high energy efficiency. In this paper, we propose a two-phase command and control (C&C) transmission scheme in a cellular-connected UAV swarm network, where the ground base station (GBS) broadcasts the common C&C message in Phase I. In Phase II, the UAVs that have successfully decoded the C&C message will relay the message to th…
Trinity College Dublin（都柏林圣三一学院）；Virginia Tech（弗吉尼亚理工大学）
Providing reliable connectivity to cellular-connected UAV can be very challenging; their performance highly depends on the nature of the surrounding environment, such as density and heights of the ground BSs. On the other hand, tall buildings might block undesired interference signals from ground BSs, thereby improving the connectivity between the UAVs and their serving BSs. To address the connectivity of UAVs in such environments, this paper proposes a RL algorithm to dynamically optimise the height of a UAV as it moves through the environment, with the goal of increasing the throughput or sp…
Unmanned Aerial Vehicles (UAVs) promise to become an intrinsic part of next generation communications, as they can be deployed to provide wireless connectivity to ground users to supplement existing terrestrial networks. The majority of the existing research into the use of UAV access points for cellular coverage considers rotary-wing UAV designs (i.e. quadcopters). However, we expect fixed-wing UAVs to be more appropriate for connectivity purposes in scenarios where long flight times are necessary (such as for rural coverage), as fixed-wing UAVs rely on a more energy-efficient form of flight …
Unmanned Aerial Vehicles (UAVs) have attracted considerable research interest recently. Especially when it comes to the realm of Internet of Things, the UAVs with Internet connectivity are one of the main demands. Furthermore, the energy constraint i.e. battery limit is a bottle-neck of the UAVs that can limit their applications. We try to address and solve the energy problem. Therefore, a path planning method for a cellular-connected UAV is proposed that will enable the UAV to plan its path in an area much larger than its battery range by getting recharged in certain positions equipped with p…
Southeast University（东南大学）；Purple Mountain Laboratories（紫金山实验室）；National University of Singapore（新加坡国立大学）
Cellular-connected unmanned aerial vehicle (UAV) is a promising technology to unlock the full potential of UAVs in the future. However, how to achieve ubiquitous three-dimensional (3D) communication coverage for the UAVs in the sky is a new challenge. In this paper, we tackle this challenge by a new coverage-aware navigation approach, which exploits the UAV's controllable mobility to design its navigation/trajectory to avoid the cellular BSs' coverage holes while accomplishing their missions. We formulate an UAV trajectory optimization problem to minimize the weighted sum of its mission comple…
A novel framework is proposed for cellular offloading with the aid of multiple unmanned aerial vehicles (UAVs), while the non-orthogonal multiple access (NOMA) technique is employed at each UAV to further improve the spectrum efficiency of the wireless network. The optimization problem of joint three-dimensional (3D) trajectory design and power allocation is formulated for maximizing the throughput. Since ground mobile users are considered as roaming continuously, the UAVs need to be re-deployed timely based on the movement of users. In an effort to solve this pertinent dynamic problem, a K-me…
In this letter, we study the energy efficiency (EE) optimisation of unmanned aerial vehicles (UAVs) providing wireless coverage to static and mobile ground users. Recent multi-agent reinforcement learning approaches optimise the system's EE using a 2D trajectory design, neglecting interference from nearby UAV cells. We aim to maximise the system's EE by jointly optimising each UAV's 3D trajectory, number of connected users, and the energy consumed, while accounting for interference. Thus, we propose a cooperative Multi-Agent Decentralised Double Deep Q-Network (MAD-DDQN) approach. Our approach…
Stevens Institute of Technology（斯蒂文斯理工学院）；University of Carthage（迦太基大学）
In this paper, we propose an autonomous UAV path planning framework using deep reinforcement learning approach. The objective is to employ a self-trained UAV as a flying mobile unit to reach spatially distributed moving or static targets in a given three dimensional urban area. In this approach, a Deep Deterministic Policy Gradient (DDPG) with continuous action space is designed to train the UAV to navigate through or over the obstacles to reach its assigned target. A customized reward function is developed to minimize the distance separating the UAV and its destination while penalizing collis…
Because of disaster, terrestrial base stations (TBS) would be partly crashed. Some user equipments (UE) would be unserved. Deploying unmanned aerial vehicles (UAV) as aerial base stations is a method to cover UEs quickly. But existing methods solely refer to the coverage of UAVs. In those scenarios, they focus on the deployment of UAVs in the post-disaster area where all TBSs do not work any longer. There is limited research about the combination of available TBSs and UAVs. We propose the method to deploy UAVs cooperating with available TBSs as aerial base stations. And improve the coverage by…
Miami University（迈阿密大学）；Illinois Institute of Technology（伊利诺伊理工学院）
Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one of the major research interests in UAV based networking. Yet few existing works have focused on how the network should react around the timing when the UAV lineup is changed. In this work, we study proactive self-remedy of energy-constrained UAV networks when one or more UAVs are short of energy and about to quit for charging. We target at an energy-aware optimal UAV control policy which proactively relocates the UAVs when any UAV is about to quit the network, rather than passively dispatches the remaining UAVs after the …
Routing Recovery for UAV Networks with Deliberate Attacks: A Reinforcement Learning based Approach
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Xidian University（西安电子科技大学）；China Academy of Railway Sciences Corporation Limited（中国铁道科学研究院集团有限公司）
Beihang University（北京航空航天大学）；Empire College London（伦敦帝国学院）；University at Buffalo（布法罗大学）
Unmanned Aerial vehicles (UAVs) are widely used as network processors in mobile networks, but more recently, UAVs have been used in Mobile Edge Computing as mobile servers. However, there are significant challenges to use UAVs in complex environments with obstacles and cooperation between UAVs. We introduce a new multi-UAV Mobile Edge Computing platform, which aims to provide better Quality-of-Service and path planning based on reinforcement learning to address these issues. The contributions of our work include: 1) optimizing the quality of service for mobile edge computing and path planning …
This paper presents an equivariant reinforcement learning framework for quadrotor unmanned aerial vehicles. Successful training of reinforcement learning often requires numerous interactions with the environments, which hinders its applicability especially when the available computational resources are limited, or when there is no reliable simulation model. We identified an equivariance property of the quadrotor dynamics such that the dimension of the state required in the training is reduced by one, thereby improving the sampling efficiency of reinforcement learning substantially. This is ill…
Safety is the primary concern when it comes to air traffic. In-flight safety between Unmanned Aircraft Vehicles (UAVs) is ensured through pairwise separation minima, utilizing conflict detection and resolution methods. Existing methods mainly deal with pairwise conflicts, however due to an expected increase in traffic density, encounters with more than two UAVs are likely to happen. In this paper, we model multi-UAV conflict resolution as a multi-agent reinforcement learning problem. We implement an algorithm based on graph neural networks where cooperative agents can communicate to jointly ge…
Korea University（高丽大学）；Deakin University（迪肯大学）；University of Oulu（奥卢大学）
In this article, we study the problem of air-to-ground ultra-reliable and low-latency communication (URLLC) for a moving ground user. This is done by controlling multiple unmanned aerial vehicles (UAVs) in real time while avoiding inter-UAV collisions. To this end, we propose a novel multi-agent deep reinforcement learning (MADRL) framework, coined a graph attention exchange network (GAXNet). In GAXNet, each UAV constructs an attention graph locally measuring the level of attention to its neighboring UAVs, while exchanging the attention weights with other UAVs so as to reduce the attention mis…
Korea University（高丽大学）；Mofl Inc.（Mofl公司）；Hallym University（翰林大学）；University of Central Florida（中佛罗里达大学）；Ajou University（亚洲大学）
CCTV-based surveillance using unmanned aerial vehicles (UAVs) is considered a key technology for security in smart city environments. This paper creates a case where the UAVs with CCTV-cameras fly over the city area for flexible and reliable surveillance services. UAVs should be deployed to cover a large area while minimize overlapping and shadow areas for a reliable surveillance system. However, the operation of UAVs is subject to high uncertainty, necessitating autonomous recovery systems. This work develops a multi-agent deep reinforcement learning-based management scheme for reliable indus…
As an emerging field of aerial robotics, Unmanned Aerial Vehicles (UAVs) have gained significant research interest within the wireless networking research community. As soon as national legislations allow UAVs to fly autonomously, we will see swarms of UAV populating the sky of our smart cities to accomplish different missions: parcel delivery, infrastructure monitoring, event filming, surveillance, tracking, etc. The UAV ecosystem can benefit from existing 5G/B5G cellular networks, which can be exploited in different ways to enhance UAV communications. Because of the inherent characteristics …
Unmanned aerial vehicles (UAVs) can provide an effective solution for improving the coverage, capacity, and the overall performance of terrestrial wireless cellular networks. In particular, UAV-assisted cellular networks can meet the stringent performance requirements of the fifth generation new radio (5G NR) applications. In this paper, the problem of energy-efficient resource allocation in UAV-assisted cellular networks is studied under the reliability and latency constraints of 5G NR applications. The framework of ruin theory is employed to allow solar-powered UAVs to capture the dynamics o…
In the current unmanned aircraft systems (UASs) for sensing services, unmanned aerial vehicles (UAVs) transmit their sensory data to terrestrial mobile devices over the unlicensed spectrum. However, the interference from surrounding terminals is uncontrollable due to the opportunistic channel access. In this paper, we consider a cellular Internet of UAVs to guarantee the Quality-of-Service (QoS), where the sensory data can be transmitted to the mobile devices either by UAV-to-Device (U2D) communications over cellular networks, or directly through the base station (BS). Since UAVs' sensing and …
Cellular network-connected unmanned aerial vehicles (UAVs) experience different radio propagation conditions than radio nodes on the ground. Therefore, it has become critical to investigate the performance of aerial radios, both theoretically and through field trials. In this paper, we consider low-altitude aerial nodes that are served by an experimental cellular network. We provide a detailed description of the hardware and software components needed for establishing a broadband wireless testbed for UAV communications research using software radios. Results show that a testbed for innovation …
This letter proposes an analytical framework to evaluate the coverage performance of a cellular-connected unmanned aerial vehicle (UAV) network in which UAV user equipments (UAV-UEs) are equipped with directional antennas and move according to a three-dimensional (3D) mobility model. The ground base stations (GBSs) equipped with practical down-tilted antennas are distributed according to a Poisson point process (PPP). With tools from stochastic geometry, we derive the handover probability and coverage probability of a random UAV-UE under the strongest average received signal strength (RSS) ass…
The rapid growth of consumer Unmanned Aerial Vehicles (UAVs) is creating promising new business opportunities for cellular operators. On the one hand, UAVs can be connected to cellular networks as new types of user equipment, therefore generating significant revenues for the operators that can guarantee their stringent service requirements. On the other hand, UAVs offer the unprecedented opportunity to realize UAV-mounted flying base stations that can dynamically reposition themselves to boost coverage, spectral efficiency, and user quality of experience. Indeed, the standardization bodies are…
Cellular networks are expected to be the main communication infrastructure to support the expanding applications of Unmanned Aerial Vehicles (UAVs). As these networks are deployed to serve ground User Equipment (UES), several issues need to be addressed to enhance cellular UAVs'services.In this paper, we propose a realistic communication model on the downlink,and we show that the Quality of Service (QoS)for the users is affected by the number of interfering BSs and the impact they cause. The joint problem of sub-carrier and power allocation is therefore addressed. Given its complexity, which i…
Beyond visual line-of-sight connectivity is key for use cases of unmanned aerial vehicles (UAVs) such as package delivery, infrastructure inspection, and rescue missions. Cellular networks stand ready to support flying UAVs by providing wide-area, quality, and secure connectivity for UAV operations. Ensuring reliable connections in the presence of UAV movements is important for safety control and operations of UAVs. With increasing height above the ground, the radio environment changes. Using terrestrial cellular networks to provide connectivity to the UAVs moving in the sky may face new chall…
Unmanned aerial vehicles (UAVs) are expected to coexist with conventional terrestrial cellular networks and become an important component to support high rate transmissions. This paper presents an analytical framework for evaluating the throughput performance of a downlink two-tier heterogeneous network. Considering the minimum distance constraint among UAVs, Matérn hardcore point process (MHP) is utilized to model the locations of UAVs. The locations of terrestrial base stations (BSs) are modeled by Poisson point process (PPP). Tools of stochastic geometry are invoked to derive tractable expr…
National Yang Ming Chiao Tung University（国立阳明交通大学）；Mississippi State University（密西西比州立大学）；Texas Tech University（德克萨斯理工大学）
Federated learning (FL) is a promising distributed learning technique particularly suitable for wireless learning scenarios since it can accomplish a learning task without raw data transportation so as to preserve data privacy and lower network resource consumption. However, current works on FL over wireless networks do not profoundly study the fundamental performance of FL over wireless networks that suffers from communication outage due to channel impairment and network interference. To accurately exploit the performance of FL over wireless networks, this paper proposes a novel intermittent …
In this paper, we consider an Internet of unmanned aerial vehicles (UAVs) over cellular networks, where UAVs work as aerial users to collect various sensory data, and send the collected data to their transmission destinations over cellular links. Unlike the terrestrial users in the conventional cellular networks, different UAVs have various communication requirements due to their sensing applications, and a more flexible communication framework is in demand. To tackle this problem, we propose a UAV-to-Everything (U2X) networking, which enables the UAVs to adjust their communication modes full …
This paper aims to propose a three-dimensional (3D) point process that can be employed to generally deploy unmanned aerial vehicles (UAVs) in a large-scale cellular network and tractably analyze the fundamental network-wide performances of the network. This 3D point process is devised based on a 2D marked Poisson point process in which each point and its random mark uniquely correspond to the projection and the altitude of each point in the 3D point process, respectively. We elaborate on some important statistical properties of the proposed 3D point process and use them to tractably analyze th…
A novel framework is proposed for cellular offloading with the aid of multiple unmanned aerial vehicles (UAVs), while non-orthogonal multiple access (NOMA) technique is employed at each UAV to further improve the spectrum efficiency of the wireless network. The optimization problem of joint three-dimensional (3D) trajectory design and power allocation is formulated for maximizing the throughput. In an effort to solve this pertinent dynamic problem, a K-means based clustering algorithm is first adopted for periodically partitioning users. Afterward, a mutual deep Q-network (MDQN) algorithm is p…
The growing use of aerial user equipments (UEs) in various applications requires ubiquitous and reliable connectivity for safe control and data exchange between these devices and ground stations. Key questions that need to be addressed when planning the deployment of aerial UEs are whether the cellular network is a suitable candidate for enabling such connectivity, and how the inclusion of aerial UEs might impact the overall network efficiency. This paper provides an in-depth analysis of user and network level performance of a cellular network that serves both unmanned aerial vehicles (UAVs) a…
Airborne base stations (carried by drones) have a great potential to enhance coverage and capacity of cellular networks. Multiple scenarios and use cases will highly benefit from such technology such as (i) offloading terrestrial base stations (BSs) in dense and urban areas, and (ii) providing coverage for rural areas. However, one of the main challenges facing the deployment of airborne BSs is the limited available energy at the drone, which limits the flight time. In fact, most of the currently used unmanned aerial vehicles (UAVs) can only operate for one hour maximum. This limits the perfor…
Cellular-based Unmanned Aerial Vehicle (UAV) systems are a promising paradigm to provide reliable and fast Beyond Visual Line of Sight (BVLoS) communication services for UAV operations. However, such systems are facing a serious GPS spoofing threat for UAV's position. To enable safe and secure UAV navigation BVLoS, this paper proposes a cellular network assisted UAV position monitoring and anti-GPS spoofing system, where deep learning approach is used to live detect spoofed GPS positions. Specifically, the proposed system introduces a MultiLayer Perceptron (MLP) model which is trained on the s…
What will the future of UAV cellular communications be? In this tutorial article, we address such a compelling yet difficult question by embarking on a journey from 5G to 6G and sharing a large number of realistic case studies supported by original results. We start by overviewing the status quo on UAV communications from an industrial standpoint, providing fresh updates from the 3GPP and detailing new 5G NR features in support of aerial devices. We then show the potential and the limitations of such features. In particular, we demonstrate how sub-6 GHz massive MIMO can successfully tackle cel…
Mobility management is the key feature of cellular networks. When integrating unmanned aerial vehicles (UAVs) into cellular networks, their cell association needs to be carefully managed for coexistence with other cellular users. UAVs move in three dimensions and may traverse several cells on their flight path, and so may be subject to several handovers. In order to enable research on mobility management with UAV users, this paper describes the design, implementation, and testing methodology for handover experiments with aerial users. We leverage software-defined radios (SDRs) and implement a …
The market size of Unmanned Aerial Vehicles (UAVs, a.k.a drones) can reach up to 10% of the global market value. In particular, drone delivery is one of the most attractive applications. The growing number of drones requires appropriate traffic management systems that will rely on cellular networks. However, it has been shown in the literature that these networks cannot provide reliable communication due to low coverage probability and frequent handovers. This article presents a potential solution targeting these problems while requiring no modifications of the existing infrastructure. Namely,…
Unmanned Aerial Vehicle (UAV) technology is becoming more prevalent and more diverse in its application. 5G and beyond networks must enable UAV connectivity. This will require the network operator to consider this new type of user in the planning and operation of the network. This work presents the challenges an operator will encounter and should consider in the future as UAVs become users of the network. We analyse the 3GPP specifications, the existing research literature, and a publicly available UAV connectivity dataset, to describe the challenges. We classify these challenges into network …
Unmanned aerial vehicle (UAV) assisted cellular communication is gaining significant interest recently. Although it offers several advantages over terrestrial communication, UAV communication suffers from two main shortcomings. The typical untethered UAV (uUAV) has a limited battery power supply and therefore limited flying time, and it needs an extra wireless backhaul link to connect users to the core network. In this paper, we propose the utilization of the tethered UAV (tUAV) to assist the cellular network, where the tether provides power supply and connects the tUAV to the core network thr…
Unmanned Aerial Vehicle (UAV) technology is becoming increasingly used in a variety of applications such as video surveillance and deliveries. To enable safe and efficient use of UAVs, the devices will need to be connected into cellular networks. Existing research on UAV cellular connectivity shows that UAVs encounter significant issues with existing networks, such as strong interference and antenna misalignment. In this work, we perform a novel measurement campaign of the performance of a UAV user when it connects to an experimental two-tier cellular network in two different areas of Dublin c…
Northern Arizona University（北亚利桑那大学）；Air Force Research Laboratory（空军研究实验室）；University of Oulu（奥卢大学）
The damage to cellular towers during natural and man-made disasters can disturb the communication services for cellular users. One solution to the problem is using unmanned aerial vehicles to augment the desired communication network. The paper demonstrates the design of a UAV-Assisted Imitation Learning (UnVAIL) communication system that relays the cellular users' information to a neighbor base station. Since the user equipment (UEs) are equipped with buffers with limited capacity to hold packets, UnVAIL alternates between different UEs to reduce the chance of buffer overflow, positions itsel…
Unmanned aerial vehicles (UAVs) are expected to play an important role in next generation cellular networks, acting as flying infrastructure which can serve ground users when regular infrastructure is overloaded or unavailable. As these devices are expected to operate wirelessly they will rely on an internal battery for their power supply, which will limit the amount of time they can operate over an area of interest before having to recharge. In this article, we outline three battery charging options that may be considered by a network operator and use simulations to demonstrate the performanc…
In this paper, we consider an Unmanned Aerial Vehicle (UAV)-assisted cellular system which consists of multiple UAV base stations (BSs) cooperating the terrestrial BSs. In such a heterogeneous network, for cellular operators, the problem is how to determine the appropriate number, locations, and altitudes of UAV-BSs to improve the system sumrate as well as satisfy the demands of arbitrarily flash crowds on data rates. We propose a data-driven 3D placement of UAV-BSs for providing an effective placement result with a feasible computational cost. The proposed algorithm searches for the appropria…
Using unmanned aerial vehicles (UAVs) to enhance network coverage has proven a variety of benefits compared to terrestrial counterparts. One of the commonly used mathematical tools to model the locations of the UAVs is stochastic geometry (SG). However, in the existing studies, both users and UAVs are often modeled as homogeneous point processes. In this paper, we consider an inhomogeneous Poisson point process (PPP)-based model for the locations of the users that captures the degradation in the density of active users as we move away from the town center. In addition, we propose the deploymen…
In this paper, we study the path planning for a cellular-connected unmanned aerial vehicle (UAV) to minimize its flying distance from given initial to final locations, while ensuring a target link quality in terms of the large-scale channel gain with each of its associated ground base stations (GBSs) during the flight. To this end, we propose the use of radio map that provides the information on the large-scale channel gains between each GBS and uniformly sampled locations on a three-dimensional (3D) grid over the region of interest, which are assumed to be time-invariant due to the generally …
