In this paper, we design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed to improve the data freshness and connectivity to the Internet of Things (IoT) devices. First, we formulate an energy-efficient trajectory optimization problem in which the objective is to maximize the energy efficiency by optimizing the UAV-BS trajectory policy. We also incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS. Second, we propose an agile deep reinforcement learning with experience replay model to solve the formulated problem concerning the contextual constraints for the UAV-BS navigation. Moreover, the proposed approach is well-suited for solving the problem, since the state space of the problem is extremely large and finding the best trajectory policy with useful contextual features is too complex for the UAV-BSs. By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time. Finally, the simulation results illustrate the proposed approach is 3.6% and 3.13% more energy efficient than those of the greedy and baseline deep Q Network (DQN) approaches.
Unmanned aerial vehicles (UAVs) are seen as a promising technology to perform a wide range of tasks in wireless communication networks. In this work, we consider the deployment of a group of UAVs to collect the data generated by IoT devices. Specifically, we focus on the case where the collected data is time-sensitive, and it is critical to maintain its timeliness. Our objective is to optimally design the UAVs' trajectories and the subsets of visited IoT devices such as the global Age-of-Updates (AoU) is minimized. To this end, we formulate the studied problem as a mixed-integer nonlinear prog…
DEI - CNIT University of Bologna（博洛尼亚大学 DEI - CNIT）；CNR-IEIIT（意大利国家研究委员会电子、信息工程与电信研究所）；ECE Department Stony Brook University（石溪大学电子与计算机工程系）
In this paper, we study a joint detection, mapping and navigation problem for a single unmanned aerial vehicle (UAV) equipped with a low complexity radar and flying in an unknown environment. The goal is to optimize its trajectory with the purpose of maximizing the mapping accuracy and, at the same time, to avoid areas where measurements might not be sufficiently informative from the perspective of a target detection. This problem is formulated as a Markov decision process (MDP) where the UAV is an agent that runs either a state estimator for target detection and for environment mapping, and a…
Research Institute of Electrical and Computer Engineering, Hanyang University（汉阳大学电气与计算机工程研究所）；Hanyang University（汉阳大学）
In this paper, we propose SACHER (soft actor-critic (SAC) with hindsight experience replay (HER)), which constitutes a class of deep reinforcement learning (DRL) algorithms. SAC is known as an off-policy model-free DRL algorithm based on the maximum entropy framework, which outperforms earlier DRL algorithms in terms of exploration, robustness and learning performance. However, in SAC, maximizing the entropy-augmented objective may degrade the optimality of learning outcomes. HER is known as a sample-efficient replay method that enhances the performance of off-policy DRL algorithms by allowing…
Southeast University（东南大学）；Purple Mountain Laboratories（紫金山实验室）；National University of Singapore（新加坡国立大学）
Cellular-connected unmanned aerial vehicle (UAV) is a promising technology to unlock the full potential of UAVs in the future. However, how to achieve ubiquitous three-dimensional (3D) communication coverage for the UAVs in the sky is a new challenge. In this paper, we tackle this challenge by a new coverage-aware navigation approach, which exploits the UAV's controllable mobility to design its navigation/trajectory to avoid the cellular BSs' coverage holes while accomplishing their missions. We formulate an UAV trajectory optimization problem to minimize the weighted sum of its mission comple…
Stevens Institute of Technology（斯蒂文斯理工学院）；University of Carthage（迦太基大学）
In this paper, we propose an autonomous UAV path planning framework using deep reinforcement learning approach. The objective is to employ a self-trained UAV as a flying mobile unit to reach spatially distributed moving or static targets in a given three dimensional urban area. In this approach, a Deep Deterministic Policy Gradient (DDPG) with continuous action space is designed to train the UAV to navigate through or over the obstacles to reach its assigned target. A customized reward function is developed to minimize the distance separating the UAV and its destination while penalizing collis…
This paper presents our method for enabling a UAV quadrotor, equipped with a monocular camera, to autonomously avoid collisions with obstacles in unstructured and unknown indoor environments. When compared to obstacle avoidance in ground vehicular robots, UAV navigation brings in additional challenges because the UAV motion is no more constrained to a well-defined indoor ground or street environment. Horizontal structures in indoor and outdoor environments like decorative items, furnishings, ceiling fans, sign-boards, tree branches etc., also become relevant obstacles unlike those for ground v…
Communication-Free Collective Navigation for a Swarm of UAVs via LiDAR-Based Deep Reinforcement Learning
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology（机械工程系，釜山国立科学技术研究院）；Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology（机械工程系，韩国科学技术院）
Age and Power Minimization via Meta-Deep Reinforcement Learning in UAV Networks
Centre for Wireless Communications (CWC), University of Oulu（奥卢大学无线通信中心）；CEMSE Division, King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学CEMSE学部）
In many emerging Internet of Things (IoT) applications, the freshness of the is an important design criterion. Age of Information (AoI) quantifies the freshness of the received information or status update. This work considers a setup of deployed IoT devices in an IoT network; multiple unmanned aerial vehicles (UAVs) serve as mobile relay nodes between the sensors and the base station. We formulate an optimization problem to jointly plan the UAVs' trajectory, while minimizing the AoI of the received messages and the devices' energy consumption. The solution accounts for the UAVs' battery lifet…
In this paper, we propose a joint indoor localization and navigation algorithm to enable a swarm of unmanned aerial vehicles (UAVs) to deploy in a specific spatial formation in indoor environments. In the envisioned scenario, we consider a static user acting as a central unit whose main task is to acquire all the UAV measurements carrying position-dependent information and to estimate the UAV positions when there is no existing infrastructure for positioning. Subsequently, the user exploits the estimated positions as inputs for the navigation control with the aim of deploying the UAVs in a des…
Unmanned aerial vehicles (UAVs) technique has been recognized as a promising solution in future wireless connectivity from the sky, and UAV navigation is one of the most significant open research problems, which has attracted wide interest in the research community. However, the current UAV navigation schemes are unable to capture the UAV motion and select the best UAV-ground links in real time, and these weaknesses overwhelm the UAV navigation performance. To tackle these fundamental limitations, in this paper, we merge the state-of-theart deep reinforcement learning with the UAV navigation t…
In this paper, we consider the cellular Internet of unmanned aerial vehicles (UAVs), where UAVs sense data for multiple tasks and transmit the data to the base station (BS). To quantify the "freshness" of the data at the BS, we bring in the concept of the age of information (AoI). The AoI is determined by the time for UAV sensing and that for UAV transmission, and gives rise to a trade-off within a given period. To minimize the AoI, we formulate a joint sensing time, transmission time, UAV trajectory, and task scheduling optimization problem. To solve this problem, we first propose an iterativ…
ETH Zürich（苏黎世联邦理工学院）；University of Bologna（博洛尼亚大学）
Nano-size unmanned aerial vehicles (UAVs), with few centimeters of diameter and sub-10 Watts of total power budget, have so far been considered incapable of running sophisticated visual-based autonomous navigation software without external aid from base-stations, ad-hoc local positioning infrastructure, and powerful external computation servers. In this work, we present what is, to the best of our knowledge, the first 27g nano-UAV system able to run aboard an end-to-end, closed-loop visual pipeline for autonomous navigation based on a state-of-the-art deep-learning algorithm, built upon the op…
Åbo Akademi University（奥布学术大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
With the growing popularity of Unmanned Aerial Vehicles (UAVs) for consumer applications, the number of accidents involving UAVs is also increasing rapidly. Therefore, motion safety of UAVs has become a prime concern for UAV operators. For a swarm of UAVs, a safe operation can not be guaranteed without preventing the UAVs from colliding with one another and with static and dynamically appearing, moving obstacles in the flying zone. In this paper, we present an online, collision-free path generation and navigation system for swarms of UAVs. The proposed system uses geographical locations of the…
Drones, unmanned aerial vehicles (UAVs), or unmanned aerial systems (UAS) are expected to be an important component of 5G/beyond 5G (B5G) communications. This includes their use within cellular architectures (5G UAVs), in which they can facilitate both wireless broadcast and point-to-point transmissions, usually using small UAS (sUAS). Allowing UAS to operate within airspace along with commercial, cargo, and other piloted aircraft will likely require dedicated and protected aviation spectrum at least in the near term, while regulatory authorities adapt to their use. The command and control (C2…
This paper studies an unmanned aerial vehicle (UAV)-assisted wireless network, where a UAV is dispatched to gather information from ground sensor nodes (SN) and transfer the collected data to the depot. The information freshness is captured by the age of information (AoI) metric, whilst the energy consumption of the UAV is seen as another performance criterion. Most importantly, the AoI and energy efficiency are inherently competing metrics, since decreasing the AoI requires the UAV returning to the depot more frequently, leading to a higher energy consumption. To this end, we design UAV paths…
Recently, unmanned aerial vehicles (UAVs) have been receiving significant attention due to their wide range of potential application areas. To support UAV use cases with beyond visual line of sight (BVLOS) and autonomous flights, cellular networks can serve as ground connectivity points, and they can provide remote control and payload communication for UAV links. However, there are limited data sets to study the coverage of cellular technologies for UAV flights at different altitudes and develop machine learning (ML) techniques for improving UAV communication and navigation. In this article, w…
In cellular-connected unmanned aerial vehicle (UAV) network, a minimization problem on the weighted sum of time cost and expected outage duration is considered. Taking advantage of UAV's adjustable mobility, an intelligent UAV navigation approach is formulated to achieve the aforementioned optimization goal. Specifically, after mapping the navigation task into a Markov decision process (MDP), a deep reinforcement learning (DRL) solution with novel quantum-inspired experience replay (QiER) framework is proposed to help the UAV find the optimal flying direction within each time slot, and thus th…
In this paper, we focus on the potential use of unmanned aerial vehicles (UAVs) for search and rescue (SAR) missions in GPS-denied indoor environments. We consider the problem of navigating a UAV to a wireless signal source, e.g., a smartphone or watch owned by a victim. We assume that the source periodically transmits RF signals to nearby wireless access points. Received signal strength (RSS) at the UAV, which is a function of the UAV and source positions, is fed to a Q-learning algorithm and the UAV is navigated to the vicinity of the source. Unlike the traditional location-based Q-learning …
In this paper, we investigate a private and cache-enabled unmanned aerial vehicle (UAV) network for content provision. Aiming at delivering fresh, fair, and energy-efficient content files to terrestrial users, we formulate a joint UAV caching, UAV trajectory, and UAV transmit power optimization problem. This problem is confirmed to be a sequential decision problem with mixed-integer non-convex constraints, which is intractable directly. To this end, we propose a novel algorithm based on the techniques of subproblem decomposition and convex approximation. Particularly, we first propose to decom…
The use of Unmanned Aerial Vehicles (UAVs) for collecting data from remotely located sensor systems is emerging. The data can be time-sensitive and require to be transmitted to a data processing center. However, planning the trajectory of a collaborative UAV swarm depends on multi-fold constraints, such as data collection requirements, UAV maneuvering capacity, and budget limitation. Since a UAV may fail or be compromised, it is important to provide necessary resilience to such contingencies, thus ensuring data security. It is important to provide the UAVs with efficient spatio-temporal trajec…
