This paper studies the traffic monitoring problem in a road network using a team of aerial robots. The problem is challenging due to two main reasons. First, the traffic events are stochastic, both temporally and spatially. Second, the problem has a non-homogeneous structure as the traffic events arrive at different locations of the road network at different rates. Accordingly, some locations require more visits by the robots compared to other locations. To address these issues, we define an uncertainty metric for each location of the road network and formulate a path planning problem for the aerial robots to minimize the network's average uncertainty. We express this problem as a partially observable Markov decision process (POMDP) and propose a distributed and scalable algorithm based on deep reinforcement learning to solve it. We consider two different scenarios depending on the communication mode between the agents (aerial robots) and the traffic management center (TMC). The first scenario assumes that the agents continuously communicate with the TMC to send/receive real-time information about the traffic events. Hence, the agents have global and real-time knowledge of the environment. However, in the second scenario, we consider a challenging setting where the observation of the aerial robots is partial and limited to their sensing ranges. Moreover, in contrast to the first scenario, the information exchange between the aerial robots and the TMC is restricted to specific time instances. We evaluate the performance of our proposed algorithm in both scenarios for a real road network topology and demonstrate its functionality in a traffic monitoring system.
This work presents an efficient and implementable solution to the problem of joint task allocation and path planning in a multi-UAV platform deployed for biometric data collection in-the-wild. The sensing requirement associated with the task gives rise to an uncanny variant of the traditional vehicle routing problem with coverage/sensing constraints. As is the case in several multi-robot path-planning problems, our problem reduces to an $m$TSP problem. In order to tame the computational challenges associated with the problem, we propose a hierarchical solution that decouples the vehicle routin…
Path Planning Optimisation for SParse, AwaRe and Cooperative Networked Aerial Robot Teams (SpArC-NARTs): Optimisation Tool and Ground Sensing Coverage Use Cases
The problem of rapid optimal coverage through the distribution a team of robots or static sensors via means of aerial drop is the topic of this work. Considering a nonholonomic (fixed-wing) aerial robot that corresponds to the carrier of a set of small holonomic (rotorcraft) aerial robots as well as static modules that are all equipped with a camera sensor, we address the problem of selecting optimal aerial drop times and configurations while the motion capabilities of the small aerial robots are also exploited to further survey their area of responsibility until they hit the ground. The overa…
In recent years, teams of robot and Unmanned Aerial Vehicles (UAVs) have been commissioned by researchers to enable accurate, online wildfire coverage and tracking. While the majority of prior work focuses on the coordination and control of such multi-robot systems, to date, these UAV teams have not been given the ability to reason about a fire's track (i.e., location and propagation dynamics) to provide performance guarantee over a time horizon. Motivated by the problem of aerial wildfire monitoring, we propose a predictive framework which enables cooperation in multi-UAV teams towards collab…
Efficient aerial data collection is important in many remote sensing applications. In large-scale monitoring scenarios, deploying a team of unmanned aerial vehicles (UAVs) offers improved spatial coverage and robustness against individual failures. However, a key challenge is cooperative path planning for the UAVs to efficiently achieve a joint mission goal. We propose a novel multi-agent informative path planning approach based on deep reinforcement learning for adaptive terrain monitoring scenarios using UAV teams. We introduce new network feature representations to effectively learn path pl…
Orientation-Aware Control and Trajectory Design for Aerial RIS-Assisted Wireless Communications
Mohammed VI Polytechnic University（穆罕默德六世理工大学）；Ricerca sul Sistema Energetico (RSE) S.p.A.（意大利能源系统研究公司（RSE））；Czech Technical University in Prague（捷克布拉格技术大学）
Autonomous Intelligent Systems Group（自主智能系统小组）；University of Bonn（波恩大学）
The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 has defined ambitious new benchmarks to advance the state-of-the-art in autonomous operation of ground-based and flying robots. This article covers our approaches to solve the two challenges that involved micro aerial vehicles (MAV). Challenge 1 required reliable target perception, fast trajectory planning, and stable control of an MAV in order to land on a moving vehicle. Challenge 3 demanded a team of MAVs to perform a search and transportation task, coined "Treasure Hunt", which required mission planning and multi-robot co…
This paper addresses the motion planning problem for a team of aerial agents under high level goals. We propose a hybrid control strategy that guarantees the accomplishment of each agent's local goal specification, which is given as a temporal logic formula, while guaranteeing inter-agent collision avoidance. In particular, by defining 3-D spheres that bound the agents' volume, we extend previous work on decentralized navigation functions and propose control laws that navigate the agents among predefined regions of interest of the workspace while avoiding collision with each other. This allows…
Arizona State University（亚利桑那州立大学）；Tufts University（塔夫茨大学）
Among the many anticipated roles for robots in the future is that of being a human teammate. Aside from all the technological hurdles that have to be overcome with respect to hardware and control to make robots fit to work with humans, the added complication here is that humans have many conscious and subconscious expectations of their teammates - indeed, we argue that teaming is mostly a cognitive rather than physical coordination activity. This introduces new challenges for the AI and robotics community and requires fundamental changes to the traditional approach to the design of autonomy. W…
Leveraging High-Fidelity Digital Models and Reinforcement Learning for Mission Engineering: A Case Study of Aerial Firefighting Under Perfect Information
Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA（弗吉尼亚理工学院与州立大学工业与系统工程系）
This technical report addresses the problem of optimal surveillance of the route followed by a dynamic parade using a team of aerial robots. The dynamic parade is considered to take place within an urban environment, it is discretized and at every iteration, the algorithm computes the best possible placing of the aerial robotic team members, subject to their camera model and the occlusions arising from the environment. As the parade route is only as well covered as its least-covered point, the optimization objective is to place the aerial robots such that they maximize the minimum coverage ove…
A mixed aerial and ground robot team, which includes both unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs), is widely used for disaster rescue, social security, precision agriculture, and military missions. However, team capability and corresponding configuration vary since robots have different motion speeds, perceiving ranges, reaching areas, and resilient capabilities to the dynamic environment. Due to heterogeneous robots inside a team and the resilient capabilities of robots, it is challenging to perform a task with an optimal balance between reasonable task allocations…
Academy for Engineering and Technology, Fudan University（复旦大学工程与技术研究院）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）；Zhejiang University（浙江大学）
Designing autonomous aerial robot team systems remains a grand challenge in robotics. Existing works in this field can be categorized as centralized and decentralized. Centralized methods suffer from scale dilemmas, while decentralized ones often lead to poor planning quality. In this paper, we propose an enhanced decentralized autonomous aerial robot team system with group planning. According to the spatial distribution of agents, the system dynamically divides the team into several groups and isolated agents. For conflicts within each group, we propose a novel coordination mechanism named gr…
University of Nevada（内华达大学）；NASA Ames Research Center（NASA艾姆斯研究中心）
Wildland fire fighting is a very dangerous job, and the lack of information of the fire front is one of main reasons that causes many accidents. Using unmanned aerial vehicle (UAV) to cover wildfire is promising because it can replace human in hazardous fire tracking and save operation costs significantly. In this paper we propose a distributed control framework designed for a team of UAVs that can closely monitor a wildfire in open space, and precisely track its development. The UAV team, designed for flexible deployment, can effectively avoid in-flight collision as well as cooperate well wit…
Czech Technical University in Prague（布拉格捷克理工大学）；University of Seville（塞维利亚大学）
This letter describes a method for autonomous aerial cinematography with distributed lighting by a team of unmanned aerial vehicles (UAVs). Although camera-carrying multi-rotor helicopters have become commonplace in cinematography, their usage is limited to scenarios with sufficient natural light or of lighting provided by static artificial lights. We propose to use a formation of unmanned aerial vehicles as a tool for filming a target under illumination from various directions, which is one of the fundamental techniques of traditional cinematography. We decompose the multi-UAV trajectory opti…
Learning to Navigate in Turbulent Flows with Aerial Robot Swarms: A Cooperative Deep Reinforcement Learning Approach
University of Pennsylvania（宾夕法尼亚大学）；Amazon Robotics（亚马逊机器人公司）；Universidad Santo Tomas（圣托马斯大学）；Bethune Cookman University（白求恩-库克曼大学）；Lehigh University（利哈伊大学）
University of Seville（塞维利亚大学）；Czech Technical University in Prague（捷克技术大学布拉格分校）
A software architecture aimed at coordinating a team of heterogeneous aerial vehicles for inspection and maintenance operations in high-voltage power line scenarios is presented in this paper. A hierarchical approach deals with high-level tasks by planning and executing complex missions requiring vehicles to support human operators. A resource-constrained problem allows distributing tasks among the team taking into account vehicles' capabilities and battery constraints. Besides, Behavior Trees (BTs) are in charge of mission execution, triggering replanning operations in case of unforeseen even…
During floods, reaching survivors in the shortest possible time is a priority for rescue teams. Given their ability to explore difficult terrain in short spans of time, Unmanned Aerial Vehicles (UAVs) have become an increasingly valuable aid to search and rescue operations. Traditionally, UAVs utilize exhaustive lawnmower exploration patterns to locate stranded survivors, without any information regarding the survivor's whereabouts. In real life disaster scenarios however, on-ground observers provide valuable information to the rescue effort, such as the survivor's last known location and head…
University of Seville（塞维利亚大学）；University of Denver（丹佛大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）
This paper addresses a synchronization problem that arises when a team of aerial robots (ARs) need to communicate while performing assigned tasks in a cooperative scenario. Each robot has a limited communication range and flies within a previously assigned closed trajectory. When two robots are close enough, a communication link may be established, allowing the robots to exchange information. The goal is to schedule the flights such that the entire system can be synchronized for maximum information exchange, that is, every pair of neighbors always visit the feasible communication link at the s…
Tsinghua University（清华大学）；Inner Mongolia University of Technology（内蒙古工业大学）；Singapore University of Technology and Design（新加坡科技设计大学）
Amphibious ground-aerial vehicles fuse flying and driving modes to enable more flexible air-land mobility and have received growing attention recently. By analyzing the existing amphibious vehicles, we highlight the autonomous fly-driving functionality for the effective uses of amphibious vehicles in complex three-dimensional urban transportation systems. We review and summarize the key enabling technologies for intelligent flying-driving in existing amphibious vehicle designs, identify major technological barriers and propose potential solutions for future research and innovation. This paper …
Service Robotics LaboratoryUniversidad Pablo de Olavide（塞维利亚大学服务机器人实验室）；Computer Vision and Aerial Robotics GroupCentre for Automation and Robotics (C.A.R.)Universidad Politécnica de Madrid (UPM-CSIC)（马德里理工大学（UPM-CSIC）自动化与机器人中心计算机视觉与空中机器人小组）；Universidad Pablo de Olavide（塞维利亚大学）；Universidad Politécnica de Madrid (UPM-CSIC)（马德里理工大学（UPM-CSIC））；Centre for Automation and Robotics (C.A.R.)（自动化与机器人中心（C.A.R.））；Computer Vision and Aerial Robotics Group（计算机视觉与空中机器人小组）；Service Robotics Laboratory（服务机器人实验室）
The paper presents a framework for fire extinguishing in an urban scenario by a team of aerial and ground robots. The system was developed to address Challenge 3 of the 2020Mohamed Bin Zayed International Robotics Challenge (MBZIRC). The challenge required to autonomously detect, locate and extinguish fires on different floors of a building, as well as in its surroundings. The multi-robot system developed consists of a heterogeneous robot team of up to three Unmanned Aerial Vehicles (UAV) and one Unmanned Ground Vehicle (UGV). We describe the main hardware and software components for UAV and U…
A Reflectance Transformation Imaging technique (RTI) realized by multi-rotor Unmanned Aerial Vehicles (UAVs) with a focus on deployment in difficult to access buildings is presented in this letter. RTI is a computational photographic method that captures a surface shape and color of a subject and enables its interactive re-lighting from any direction in a software viewer, revealing details that are not visible with the naked eye. The input of RTI is a set of images captured by a static camera, each one under illumination from a different known direction. We present an innovative approach apply…
New York University, Tandon School of Engineering（纽约大学坦登工程学院）；Technology Innovation Institute（技术创新研究院）
In this paper, we address the vision-based detection and tracking problems of multiple aerial vehicles using a single camera and Inertial Measurement Unit (IMU) as well as the corresponding perception consensus problem (i.e., uniqueness and identical IDs across all observing agents). We design several vision-based decentralized Bayesian multi-tracking filtering strategies to resolve the association between the incoming unsorted measurements obtained by a visual detector algorithm and the tracked agents. We compare their accuracy in different operating conditions as well as their scalability ac…
University of Nevada, Reno（内华达大学里诺分校）；NTNU（挪威科技大学）；ETH Zurich（苏黎世联邦理工学院）
This paper presents a novel strategy for autonomous teamed exploration of subterranean environments using legged and aerial robots. Tailored to the fact that subterranean settings, such as cave networks and underground mines, often involve complex, large-scale and multi-branched topologies, while wireless communication within them can be particularly challenging, this work is structured around the synergy of an onboard exploration path planner that allows for resilient long-term autonomy, and a multi-robot coordination framework. The onboard path planner is unified across legged and flying rob…
University of the Western Cape（西开普大学）；Cartage University（迦太基大学）
The use of Unmanned Aerial Vehicles (UAVs) in Data transport has attracted a lot of attention and applications, as a modern traffic engineering technique used in data sensing, transport, and delivery to where infrastructure is available for its interpretation. Due to UAVs constraints such as limited power lifetime, it has been necessary to assist them with ground sensors to gather local data which has to be transferred to UAVs upon visiting the sensors. The management of such ground sensor communication together with a team of flying UAVs constitutes an interesting data muling problem which st…
Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克理工大学电气工程学院）
This paper presents a system for autonomous cooperative wall building with a team of Unmanned Aerial Vehicles (UAVs). The system was developed for Challenge 2 of the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. The wall-building scenario of Challenge 2 featured an initial stack of bricks and wall structure where the individual bricks had to be placed by a team of three UAVs. The objective of the task was to maximize collected points for placing the bricks within the restricted construction time while following the prescribed wall pattern. The proposed approach uses initial…
University of Nevada, Reno（内华达大学里诺分校）；NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）
Wild-land fire fighting is a hazardous job. A key task for firefighters is to observe the "fire front" to chart the progress of the fire and areas that will likely spread next. Lack of information of the fire front causes many accidents. Using Unmanned Aerial Vehicles (UAVs) to cover wildfire is promising because it can replace humans in hazardous fire tracking and significantly reduce operation costs. In this paper we propose a distributed control framework designed for a team of UAVs that can closely monitor a wildfire in open space, and precisely track its development. The UAV team, designe…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Maryland（马里兰大学）
In this paper, we study the problem of exploring an unknown Region Of Interest (ROI) with a team of aerial robots. The size and shape of the ROI are unknown to the robots. The objective is to find a tour for each robot such that each point in the ROI must be visible from the field-of-view of some robot along its tour. In conventional exploration using ground robots, the ROI boundary is typically also as an obstacle and robots are naturally constrained to the interior of this ROI. Instead, we study the case where aerial robots are not restricted to flying inside the ROI (and can fly over the bo…
Vision Foundation Models for Domain Generalisable Cross-View Localisation in Planetary Ground-Aerial Robotic Teams
AI for Space Group and 3 Andy Thomas Centre for Space Resources, The University of Adelaide（AI空间组和安迪·托马斯太空资源中心，阿德莱德大学）；Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA（喷气推进实验室，加州理工学院，帕萨迪纳，CA 91109，美国）；California Institute of Technology（加州理工学院）
In this paper we present a cooperative multi-robot strategy to adaptively explore and sample environments that are unfavorable for humans. We propose a methodology for a team of heterogeneous robots to collaborate on information based planning for applications like sampling thermal imagery in a wildfire affected site to assist with detecting spot fires and areas of residual fires, fire mapping and monitoring fire progression or applications in marine domain for coral reef monitoring and survey. We use Gabor filter based texture classifier on aerial images from an Unmanned Aerial Vehicle (UAV) …
Swarms are highly robust systems that offer unique benefits compared to their alternatives. In this work, we propose a bio-inspired and artificial potential field-driven robot swarm control method, where the swarm formation dynamics are modeled on the basis of Markov Random Field (MRF) optimization. We integrate the internal agent-wise local interactions and external environmental influences into the MRF. The optimized formation configurations at different stages of the trajectory can be viewed as formation "shapes" which further allows us to integrate dynamics-constrained motion control of th…
For robotic systems to succeed in high risk, real-world situations, they have to be quickly deployable and robust to environmental changes, under-performing hardware, and mission subtask failures. These robots are often designed to consider a single sequence of mission events, with complex algorithms lowering individual subtask failure rates under some critical constraints. Our approach utilizes common techniques in vision and control, and encodes robustness into mission structure through outcome monitoring and recovery strategies. In addition, our system infrastructure enables rapid deploymen…
Texas A&M University（德克萨斯农工大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
In this paper, an autonomous tethered Unmanned Aerial Vehicle (UAV) is developed into a visual assistant in a marsupial co-robots team, collaborating with a tele-operated Unmanned Ground Vehicle (UGV) for robot operations in unstructured or confined environments. These environments pose extreme challenges to the remote tele-operator due to the lack of sufficient situational awareness, mostly caused by the unstructuredness and confinement, stationary and limited field-of-view and lack of depth perception from the robot's onboard cameras. To overcome these problems, a secondary tele-operated rob…
The collective operation of robots, such as unmanned aerial vehicles (UAVs) operating as a team or swarm, is affected by their individual capabilities, which in turn is dependent on their physical design, aka morphology. However, with the exception of a few (albeit ad hoc) evolutionary robotics methods, there has been very little work on understanding the interplay of morphology and collective behavior. There is especially a lack of computational frameworks to concurrently search for the robot morphology and the hyper-parameters of their behavior model that jointly optimize the collective (tea…
University of Illinois at Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）
We propose a novel scheme for surveillance of a dynamic ground convoy moving along a non-linear trajectory, by aerial agents that maintain a uniformly spaced formation on a time-varying elliptical orbit encompassing the convoy. Elliptical orbits are used as they are more economical than circular orbits for circumnavigating the group of targets in the moving convoy. The proposed scheme includes an algorithm for computing feasible elliptical orbits, a vector guidance law for agent motion along the desired orbit, and a cooperative strategy to control the speeds of the aerial agents in order to qu…
We present a framework for a ground-aerial robotic team to explore large, unstructured, and unknown environments. In such exploration problems, the effectiveness of existing exploration-boosting heuristics often scales poorly with the environments' size and complexity. This work proposes a novel framework combining incremental frontier distribution, goal selection with Monte-Carlo view quality rendering, and an automatic-differentiable information gain measure to improve exploration efficiency. Simulated with multiple complex environments, we demonstrate that the proposed method effectively ut…
Multimodal Large Language Models-Enabled UAV Swarm: Towards Efficient and Intelligent Autonomous Aerial Systems
College of Informatics, Harbin Institute of Technology（信息学院，哈尔滨工业大学）；Key Laboratory of Forest and Grassland Fire Risk Prevention, Ministry of Emergency Management, China Fire and Rescue Institute（森林和草原火灾风险预防重点实验室，应急管理部，中国消防救援学院）；Southern University of Science and Technology（南方科技大学）
The communication system is a critical part of the system design for the autonomous UAV. It has to address different considerations, including efficiency, reliability and mobility of the UAV. In addition, a multi-UAV system requires a communication system to assist information sharing, task allocation and collaboration in a team of UAVs. In this paper, we review communication solutions for supporting a team of UAVs while considering an application in the power line inspection industry. We provide a review of candidate wireless communication technologies {for supporting communication in UAV app…
University of Zagreb（萨格勒布大学）；Faculty of Electrical Engineering and Computing（电气工程与计算学院）
In this paper, we study a cooperative aerial-ground robotic team and its application to the task of automated construction. We propose a solution for planning and coordinating the mission of constructing a wall with a predefined structure for a heterogeneous system consisting of one mobile robot and up to three unmanned aerial vehicles. The wall consists of bricks of various weights and sizes, some of which need to be transported using multiple robots simultaneously. To that end, we use hierarchical task representation to specify interrelationships between mission subtasks and employ effective…
