Unmanned aerial vehicle (UAV) swarm control has applications including target tracking, surveillance, terrain mapping, and precision agriculture. Decentralized control methods are particularly useful when the swarm is large, as centralized methods (a single command center controlling the UAVs) suffer from exponential computational complexity, i.e., the computing time to obtain the optimal control for the UAVs grow exponentially with the number of UAVs in the swarm in centralized approaches. Although many centralized control methods exist, literature lacks decentralized control frameworks with broad applicability. To address this knowledge gap, we present a novel decentralized UAV swarm control strategy using a decision-theoretic framework called decentralized Markov decision process (Dec-MDP). We build these control strategies in the context of two case studies: a) swarm formation control problem; b) swarm control for multitarget tracking. As most decision theoretic formulations suffer from the curse of dimensionality, we adapt an approximate dynamic programming method called nominal belief-state optimization (NBO) to solve the decentralized control problems approximately in both the case studies. In the formation control case study, the objective is to drive the swarm from a geographical region to another geographical region where the swarm must form a certain geometrical shape (e.g., selected location on the surface of a sphere). The motivation for studying such problems comes from data fusion applications with UAV swarms where the fusion performance depends on the strategic relative separation of the UAVs from each other. In the target tracking case study, the objective is the control the motion of the UAVs in a decentralized manner while maximizing the overall target tracking performance. Motivation for this case study comes from the surveillance applications using UAV swarms.
Safety-Critical Control for Quadrotor UAVs via Decentralized Navigation Functions
Universidad Politécnica de Madrid（马德里理工大学）；School of Science and Technology, IE University（IE大学科学与技术学院）；Centre for Automation and Robotics (CSIC-UPM)（自动化与机器人中心（西班牙国家科学研究委员会-马德里理工大学））
With the advancement in smart grid and smart energy devices, demand response becomes one of the most economic and feasible solutions to ease the load stress of the power grids during peak hours. In this work, we propose a fully decentralized ensemble control framework with consensus for demand response (DR) events and compatible control methods based on random policies. We show that under the consensus that is tailored to DR, our proposed decentralized control method yields the same optimality as the centralized control method in both myopic and multistage settings.
Indian Institute of Science（印度科学学院）；Nanyang Technological University（南洋理工大学）
This paper provides a new solution approach to a multi-player perimeter defense game, in which the intruders' team tries to enter the territory, and a team of defenders protects the territory by capturing intruders on the perimeter of the territory. The objective of the defenders is to detect and capture the intruders before the intruders enter the territory. Each defender independently senses the intruder and computes his trajectory to capture the assigned intruders in a cooperative fashion. The intruder is estimated to reach a specific location on the perimeter at a specific time. Each intru…
University of Maryland, College Park（马里兰大学帕克分校）；University of Toronto（多伦多大学）；DEVCOM Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）
We study a chance-constrained variant of the cooperative aerial-ground vehicle routing problem, in which an Unmanned Aerial Vehicle (UAV) with limited battery capacity and an Unmanned Ground Vehicle (UGV) that can also act as a mobile recharging station need to jointly accomplish a mission such as monitoring a set of points. Due to the limited battery capacity of the UAV, two vehicles sometimes have to deviate from their task to rendezvous and recharge the UAV\@. Unlike prior work that has focused on the deterministic case, we address the challenge of stochastic energy consumption of the UAV\@…
Convergence of Decentralized Actor-Critic Algorithm in General-sum Markov Games
Electrical Engineering and Computer Sciences (EECS) at UC Berkeley（加州大学伯克利分校电子工程与计算机科学系）；Operations Research and Information Engineering (ORIE) at Cornell University（康奈尔大学运筹学与信息工程系）
College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学与技术学院）
Developing the flocking behavior for a dynamic squad of fixed-wing UAVs is still a challenge due to kinematic complexity and environmental uncertainty. In this paper, we deal with the decentralized flocking and collision avoidance problem through deep reinforcement learning (DRL). Specifically, we formulate a decentralized DRL-based decision making framework from the perspective of every follower, where a collision avoidance mechanism is integrated into the flocking controller. Then, we propose a novel reinforcement learning algorithm PS-CACER for training a shared control policy for all the f…
High-Probability Nash Regret for Decentralized Learning in Markov $α$-Potential Games: Episodic and Fully Online Asynchronous Algorithms with Applications to Markov Congestion Games
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We describe the orchestration of a decentralized swarm of rotary-wing UAV-relays, augmenting the coverage and service capabilities of a terrestrial base station. Our goal is to minimize the time-average service latencies involved in handling transmission requests from ground users under Poisson arrivals, subject to an average UAV power constraint. Equipped with rate adaptation to efficiently leverage air-to-ground channel stochastics, we first derive the optimal control policy for a single relay via a semi-Markov decision process formulation, with competitive swarm optimization for UAV traject…
Dynamic one-time delivery of critical data by small and sparse UAV swarms: a model problem for MARL scaling studies
Swedish Defence Research Agency (FOI)（瑞典国防研究机构（FOI））；Chalmers University of Technology（查尔姆斯理工大学）；the University of Gothenburg, Department of Mathematical Sciences（哥德堡大学数学科学系）
