We propose a risk-aware framework for multi-robot, multi-demand assignment and planning in unknown environments. Our motivation is disaster response and search-and-rescue scenarios where ground vehicles must reach demand locations as soon as possible. We consider a setting where the terrain information is available only in the form of an aerial, georeferenced image. Deep learning techniques can be used for semantic segmentation of the aerial image to create a cost map for safe ground robot navigation. Such segmentation may still be noisy. Hence, we present a joint planning and perception framework that accounts for the risk introduced due to noisy perception. Our contributions are two-fold: (i) we show how to use Bayesian deep learning techniques to extract risk at the perception level; and (ii) use a risk-theoretical measure, CVaR, for risk-aware planning and assignment. The pipeline is theoretically established, then empirically analyzed through two datasets. We find that accounting for risk at both levels produces quantifiably safer paths and assignments.
Australian National University（澳大利亚国立大学）；University of Technology Sydney（悉尼科技大学）
This paper presents a novel dense optical-flow algorithm to solve the monocular simultaneous localization and mapping (SLAM) problem for ground or aerial robots. Dense optical flow can effectively provide the ego-motion of the vehicle while enabling collision avoidance with the potential obstacles. Existing work has not fully utilized the uncertainty of the optical flow -- at most an isotropic Gaussian density model. We estimate the full uncertainty of the optical flow and propose a new eight-point algorithm based on the statistical Mahalanobis distance. Combined with the pose-graph optimizati…
Massachusetts Institute of Technology（麻省理工学院）；Delft University of Technology（代尔夫特理工大学）
We address the problem of assigning a team of drones to autonomously capture a set desired shots of a dynamic target in the presence of obstacles. We present a two-stage planning pipeline that generates offline an assignment of drone to shots and locally optimizes online the viewpoint. Given desired shot parameters, the high-level planner uses a visibility heuristic to predict good times for capturing each shot and uses an Integer Linear Program to compute drone assignments. An online Model Predictive Control algorithm uses the assignments as reference to capture the shots. The algorithm is va…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Agents and Applied Robotics Group, IIIT Hyderabad（印度理工学院海得拉巴分校智能代理与应用机器人小组）；Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）
Autonomous motion capture (mocap) systems for outdoor scenarios involving flying or mobile cameras rely on i) a robotic front-end to track and follow a human subject in real-time while he/she performs physical activities, and ii) an algorithmic back-end that estimates full body human pose and shape from the saved videos. In this paper we present a novel front-end for our aerial mocap system that consists of multiple micro aerial vehicles (MAVs) with only on-board cameras and computation. In previous work, we presented an approach for cooperative detection and tracking (CDT) of a subject using …
Computationally tractable methods are developed for centralized goal assignment and planning of collision-free polynomial-in-time trajectories for systems of multiple aerial robots. The method first assigns robots to goals to minimize total time-in-motion based on initial trajectories. By coupling the assignment and trajectory generation, the initial motion plans tend to require only limited collision resolution. The plans are then refined by checking for potential collisions and resolving them using either start time delays or altitude assignment. Numerical experiments using both methods show…
University of Michigan（密歇根大学）；NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）
With the number of small Unmanned Aircraft Systems (sUAS) in the national airspace projected to increase in the next few years, there is growing interest in a traffic management system capable of handling the demands of this aviation sector. It is expected that such a system will involve trajectory prediction, uncertainty propagation, and path planning algorithms. In this work, we use linear covariance propagation in combination with a quadratic programming-based collision detection algorithm to rapidly validate declared flight plans. Additionally, these algorithms are combined with a Dynamic,…
Multiple multirotor Micro Aerial Vehicles sharing the same airspace require a reliable and robust collision avoidance technique. In this paper we address the problem of multi-MAV reactive collision avoidance. A model-based controller is employed to achieve simultaneously reference trajectory tracking and collision avoidance. Moreover, we also account for the uncertainty of the state estimator and the other agents position and velocity uncertainties to achieve a higher degree of robustness. The proposed approach is decentralized, does not require collision-free reference trajectory and accounts…
Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
Semantic segmentation of aerial imagery is an important tool for mapping and earth observation. However, supervised deep learning models for segmentation rely on large amounts of high-quality labelled data, which is labour-intensive and time-consuming to generate. To address this, we propose a new approach for using unmanned aerial vehicles (UAVs) to autonomously collect useful data for model training. We exploit a Bayesian approach to estimate model uncertainty in semantic segmentation. During a mission, the semantic predictions and model uncertainty are used as input for terrain mapping. A k…
We consider a pursuit-evasion problem with a heterogeneous team of multiple pursuers and multiple evaders. Although both the pursuers (robots) and the evaders are aware of each others' control and assignment strategies, they do not have exact information about the other type of agents' location or action. Using only noisy on-board sensors the pursuers (or evaders) make probabilistic estimation of positions of the evaders (or pursuers). Each type of agent use Markov localization to update the probability distribution of the other type. A search-based control strategy is developed for the pursue…
We consider scenarios where a ground vehicle plans its path using data gathered by an aerial vehicle. In the aerial images, navigable areas of the scene may be occluded due to obstacles. Naively planning paths using aerial images may result in longer paths as a conservative planner may try to avoid regions that are occluded. We propose a modular, deep learning-based framework that allows the robot to predict the existence of navigable areas in the occluded regions. Specifically, we use image inpainting methods to fill in parts of the areas that are potentially occluded, which can then be seman…
STL-Based Motion Planning and Uncertainty-Aware Risk Analysis for Human-Robot Collaboration with a Multi-Rotor Aerial Vehicle
Department of Cybernetics, Czech Technical University in Prague（捷克技术大学布拉格分校控制学系）；Department of Power Generation Technologies and Materials, Ricerca sul Sistema Energetico S.p.A.（能源系统研究股份有限公司电力生成技术与材料学系）；Robotics and Mechatronics Department, Electrical Engineering, Mathematics, and Computer Science Faculty, University of Twente（代尔夫特理工大学机器人与机电学系，电气工程、数学与计算机科学学院）；Department of Computer, Control and Management Engineering, Sapienza University of Rome（罗马萨皮恩扎大学计算机、控制与管理工程系）
Enhancing Drone Light Shows Performances: Optimal Allocation and Trajectories for Swarm Drone Formations
Aerospace Engineering Department, Faculty of Aeronautics and Astronautics, Kocaeli University（航空航天工程系，航空航天学院，科伊亚利大学）；Interdisciplinary Research Centre for Aviation and Space Exploration (IRC-ASE), KFUPM（航空与航天探索交叉研究中心（IRC-ASE），国王费萨尔大学）
Marsupial robot teams consist of carrier robots that transport and deploy multiple passenger robots, such as a team of ground robots that carry and deploy multiple aerial robots, to rapidly explore complex environments. We specifically address the problem of planning the deployment times and locations of the carrier robots to best meet the objectives of a mission while reasoning over uncertain future observations and rewards. While prior work proposed optimal, polynomial-time solutions to single-carrier robot systems, the multiple-carrier robot deployment problem is fundamentally harder as it …
This paper considers the problem of assigning multiple mobile robots to goals on transport networks with uncertain information about travel times. Our aim is to produce optimal assignments, such that the average waiting time at destinations is minimized. Since noisy travel time estimates result in sub-optimal assignments, we propose a method that offers resilience to uncertainty by making use of redundant robots. However, solving the redundant assignment problem optimally is strongly NP-hard. Hence, we exploit structural properties of our mathematical problem formulation to propose a polynomia…
We provide a framework for the assignment of multiple robots to goal locations, when robot travel times are uncertain. Our premise is that time is the most valuable asset in the system. Hence, we make use of redundant robots to counter the effect of uncertainty and minimize the average waiting time at destinations. We apply our framework to transport networks represented as graphs, and consider uncertainty in the edge costs (i.e., travel time). Since solving the redundant assignment problem is strongly NP-hard, we exploit structural properties of our problem to propose a polynomial-time soluti…
Yale University（耶鲁大学）；SDU UAS（南丹麦大学无人机系统中心）；University of Granada（格拉纳达大学）；University of Southern Denmark（南丹麦大学）
In this paper, we present an online planning-scheduling approach for battery-powered autonomous aerial robots. The approach consists of simultaneously planning a coverage path and scheduling onboard computational tasks. We further derive a novel variable coverage motion robust to airborne constraints and an empirically motivated energy model. The model includes the energy contribution of the schedule based on an automatic computational energy modeling tool. Our experiments show how an initial flight plan is adjusted online as a function of the available battery, accounting for uncertainty. Our…
This work presents a two part framework for online planning and execution of dynamic aerial motions on a quadruped robot. Motions are planned via a centroidal momentum-based nonlinear optimization that is general enough to produce rich sets of novel dynamic motions based solely on the user-specified contact schedule and desired launch velocity of the robot. Since this nonlinear optimization is not tractable for real-time receding horizon control, motions are planned once via nonlinear optimization in preparation of an aerial motion and then tracked continuously using a variational-based optima…
Self-Organizing Aerial Swarm Robotics for Resilient Load Transportation : A Table-Mechanics-Inspired Approach
School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）；Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong（香港中文大学机械与自动化工程系）
University of Pennsylvania（宾夕法尼亚大学）；UC San Diego（加州大学圣地亚哥分校）
Search-based motion planning has been used for mobile robots in many applications. However, it has not been fully developed and applied for planning full state trajectories of Micro Aerial Vehicles (MAVs) due to their complicated dynamics and the requirement of real-time computation. In this paper, we explore a search-based motion planning framework that plans dynamically feasible, collision-free, and resolution optimal and complete trajectories. This paper extends the search-based planning approach to address three important scenarios for MAVs navigation: (i) planning safe trajectories in the…
A Signal Temporal Logic Approach for Task-Based Coordination of Multi-Aerial Systems: a Wind Turbine Inspection Case Study
Ricerca sul Sistema Energetico (RSE) S.p.A.（意大利能源系统研究公司（RSE股份公司））；University of Seville（塞维利亚大学）；Italian National Agency for New Technologies, Energy and Sustainable Economic Development (ENEA)（意大利新技术、能源与可持续经济发展局）；University of Zagreb（萨格勒布大学）
In this paper, we present an on-board vision-based approach for avoidance of moving obstacles in dynamic environments. Our approach relies on an efficient obstacle detection and tracking algorithm based on depth image pairs, which provides the estimated position, velocity and size of the obstacles. Robust collision avoidance is achieved by formulating a chance-constrained model predictive controller (CC-MPC) to ensure that the collision probability between the micro aerial vehicle (MAV) and each moving obstacle is below a specified threshold. The method takes into account MAV dynamics, state e…
Australian Center for Field Robotics, the University of Sydney（悉尼大学澳大利亚野外机器人中心）
Centimeter level globally accurate and consistent maps for autonomous vehicles navigation has long been achieved by on board real-time kinematic(RTK)-GPS in open areas. However when dealing with urban environments, GPS will experience multipath and blockage in urban canyon, under bridges, inside tunnels and in underground environments. In this paper we present strategies to efficiently register local maps in geographical coordinate systems through the tactical integration of GPS and information extracted from precisely geo-referenced high resolution aerial orthogonal imagery. Dense lidar point…
School of Electrical Engineering and Computer Science, The University of Queensland（昆士兰大学电气工程与计算机科学学院）；CSIRO Robotics, Data61（联邦科学与工业研究组织机器人学与Data61研究院）
Collision avoidance requires tradeoffs in planning time horizons. Depending on the planner, safety cannot always be guaranteed in uncertain environments given map updates. To mitigate situations where the planner leads the vehicle into a state of collision or the vehicle reaches a point where no trajectories are feasible, we propose a continuous collision checking algorithm. The imminent collision checking system continuously monitors vehicle safety, and plans a safe trajectory that leads the vehicle to a stop within the observed map. We test our proposed pipeline alongside a teleoperated navi…
Robust Planning and Control of Omnidirectional MRAVs for Aerial Communications in Wireless Networks
Ricerca sul Sistema Energetico（能源系统研究）；Czech Technical University in Prague（布拉格捷克技术大学）；College of Computing, Mohammed VI Polytechnic University（穆莱·伊斯梅尔polytechnic大学计算机学院）
Modular Adaptive Aerial Manipulation under Unknown Dynamic Coupling Forces
The University of Manchester（曼彻斯特大学）；Czech Technical University in Prague（布拉格捷克技术大学）；International Institute of Information Technology Hyderabad（海德拉巴国际信息技术学院）；Southeast University（东南大学）
BIM-Discrepancy-Driven Active Sensing for Risk-Aware UAV-UGV Navigation
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA, USA（电气与计算机工程系，加州大学圣地亚哥分校）；Department of Civil, Construction, and Environmental Engineering, San Diego State University, San Diego, CA, USA（土木、建设与环境工程系，圣地亚哥州立大学）
There is a strong demand for covering a large area autonomously by multiple UAVs (Unmanned Aerial Vehicles) supported by a ground vehicle. Limited by UAVs' battery life and communication distance, complete coverage of large areas typically involves multiple take-offs and landings to recharge batteries, and the transportation of UAVs between operation areas by a ground vehicle. In this paper, we introduce a novel large-area-coverage planning framework which collectively optimizes the paths for aerial and ground vehicles. Our method first partitions a large area into sub-areas, each of which a g…
This work presents an efficient and implementable solution to the problem of joint task allocation and path planning in a multi-UAV platform deployed for biometric data collection in-the-wild. The sensing requirement associated with the task gives rise to an uncanny variant of the traditional vehicle routing problem with coverage/sensing constraints. As is the case in several multi-robot path-planning problems, our problem reduces to an $m$TSP problem. In order to tame the computational challenges associated with the problem, we propose a hierarchical solution that decouples the vehicle routin…
QP-Based Control of an Underactuated Aerial Manipulator under Constraints
Laboratoire d'automatique de Tlemcen（特马森自动化实验室）；Laboratoire d’Ingénierie des Systèmes de Versailles, Université de Versailles Saint Quentin en Yvelines, France（巴黎-萨克雷大学系统工程实验室）；Joint Robotics Laboratory, National Institute of Advanced Industrial Science and Technology (AIST)（先进工业科学技术国家研究所机器人联合实验室）
Norwegian University of Science and Technology (NTNU)（挪威科技大学（NTNU））；Voliro AG（Voliro公司）；ETH Zurich（苏黎世联邦理工学院）
This paper presents a review of the design and application of model predictive control strategies for Micro Aerial Vehicles and specifically multirotor configurations such as quadrotors. The diverse set of works in the domain is organized based on the control law being optimized over linear or nonlinear dynamics, the integration of state and input constraints, possible fault-tolerant design, if reinforcement learning methods have been utilized and if the controller refers to free-flight or other tasks such as physical interaction or load transportation. A selected set of comparison results are…
Universidade Federal do Rio Grande (FURG)（里奥格兰德联邦大学（FURG））；Universidad Tecnológica del Uruguay (UTEC)（乌拉圭技术大学（UTEC））
Unmanned Aerial Vehicles (UAV) have been standing out due to the wide range of applications in which they can be used autonomously. However, they need intelligent systems capable of providing a greater understanding of what they perceive to perform several tasks. They become more challenging in complex environments since there is a need to perceive the environment and act under environmental uncertainties to make a decision. In this context, a system that uses active perception can improve performance by seeking the best next view through the recognition of targets while displacement occurs. T…
This work presents an online learning-based control method for improved trajectory tracking of unmanned aerial vehicles using both deep learning and expert knowledge. The proposed method does not require the exact model of the system to be controlled, and it is robust against variations in system dynamics as well as operational uncertainties. The learning is divided into two phases: offline (pre-)training and online (post-)training. In the former, a conventional controller performs a set of trajectories and, based on the input-output dataset, the deep neural network (DNN)-based controller is t…
Optimal transport (OT) is a framework that can guide the design of efficient resource allocation strategies in a network of multiple sources and targets. This paper applies discrete OT to a swarm of UAVs in a novel way to achieve appropriate task allocation and execution. Drone swarm deployments already operate in multiple domains where sensors are used to gain knowledge of an environment [1]. Use cases such as, chemical and radiation detection, and thermal and RGB imaging create a specific need for an algorithm that considers parameters on both the UAV and waypoint side and allows for updatin…
This paper studies the traffic monitoring problem in a road network using a team of aerial robots. The problem is challenging due to two main reasons. First, the traffic events are stochastic, both temporally and spatially. Second, the problem has a non-homogeneous structure as the traffic events arrive at different locations of the road network at different rates. Accordingly, some locations require more visits by the robots compared to other locations. To address these issues, we define an uncertainty metric for each location of the road network and formulate a path planning problem for the …
The recent development of novel aerial vehicles capable of physically interacting with the environment leads to new applications such as contact-based inspection. These tasks require the robotic system to exchange forces with partially-known environments, which may contain uncertainties including unknown spatially-varying friction properties and discontinuous variations of the surface geometry. Finding a control strategy that is robust against these environmental uncertainties remains an open challenge. This paper presents a learning-based adaptive control strategy for aerial sliding tasks. In…
