Heterogeneous air-ground robot teams combine complementary sensing modalities, mobility characteristics, and spatial viewpoints that can significantly enhance perception in complex outdoor environments. However, progress in multi-robot collaborative perception has been constrained by the lack of real-world datasets featuring overlapping multi-modal observations from platforms operating in unstructured terrain. We present the \textbf{AGT-CV} (\textbf{A}erial-\textbf{G}round \textbf{T}eam \textbf{C}ross-\textbf{V}iew) dataset, a real-world multi-robot collaborative perception dataset collected using a Clearpath Husky UGV and an Autel EVO~II UAV across diverse unstructured environments, including forest trails, rocky paths, muddy terrain, snow piles, and grass-covered fields. The ground platform provides 3D LiDAR, stereo camera, IMU, and GPS data, while the aerial platform contributes RGB imagery, thermal/infrared observations, and GPS from a complementary overhead viewpoint, allowing for rich cross-modal and cross-view perception. The dataset is collected in 4 unique environments, with over 13,000 synchronized frames across approximately 29 minutes of operation, and includes both SAM~3-based zero-shot segmentation and almost 8,000 manually labeled images. A unique aspect of the dataset is its early-spring collection period, during which sparse tree canopies allow the aerial robot to partially observe the ground robot and terrain through the trees, allowing for occlusion-aware collaborative perception. Unlike prior multi-robot datasets that primarily focus on SLAM or simulated cooperative driving, AGT-CV is specifically designed to support research on cross-view perception, air-ground viewpoint fusion, terrain-aware perception, and collaborative scene understanding in real off-road environments.
Performing rescuing and surveillance operations with autonomous ground and aerial vehicles become more and more apparent task. Involving unmanned robot systems allows making these operations more efficient, safe and reliable especially in hazardous areas. This work is devoted to the development of a cost-efficient micro aerial vehicle in a quadrocopter shape for developmental purposes within indoor scenarios. It has been constructed with off-the-shelf components available for mini helicopters. Additional sensors and electronics are incorporated into this aerial vehicle to stabilize its flight …
This research aims at developing path and motion planning algorithms for a tethered Unmanned Aerial Vehicle (UAV) to visually assist a teleoperated primary robot in unstructured or confined environments. The emerging state of the practice for nuclear operations, bomb squad, disaster robots, and other domains with novel tasks or highly occluded environments is to use two robots, a primary and a secondary that acts as a visual assistant to overcome the perceptual limitations of the sensors by providing an external viewpoint. However, the benefits of using an assistant have been limited for at le…
We present a method for learning to drive on smooth terrain while simultaneously avoiding collisions in challenging off-road and unstructured outdoor environments using only visual inputs. Our approach applies a hybrid model-based and model-free reinforcement learning method that is entirely self-supervised in labeling terrain roughness and collisions using on-board sensors. Notably, we provide both first-person and overhead aerial image inputs to our model. We find that the fusion of these complementary inputs improves planning foresight and makes the model robust to visual obstructions. Our …
Carnegie Mellon University（卡内基梅隆大学）；University of Washington（华盛顿大学）
The use of drones for aerial cinematography has revolutionized several applications and industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely controlling a drone while filming a moving target usually requires multiple expert human operators; hence the need for an autonomous cinematographer. Current approaches have severe real-life limitations such as requiring fully scripted scenes, high-precision motion-capture systems or GPS tags to localize targets, and prior maps of the environment to avoid obstacles and plan for occlusion. …
Increased growth in the global Unmanned Aerial Vehicles (UAV) (drone) industry has expanded possibilities for fully autonomous UAV applications. A particular application which has in part motivated this research is the use of UAV in wide area search and surveillance operations in unstructured outdoor environments. The critical issue with such environments is the lack of structured features that could aid in autonomous flight, such as road lines or paths. In this paper, we propose an End-to-End Multi-Task Regression-based Learning approach capable of defining flight commands for navigation and …
We present a framework for a ground-aerial robotic team to explore large, unstructured, and unknown environments. In such exploration problems, the effectiveness of existing exploration-boosting heuristics often scales poorly with the environments' size and complexity. This work proposes a novel framework combining incremental frontier distribution, goal selection with Monte-Carlo view quality rendering, and an automatic-differentiable information gain measure to improve exploration efficiency. Simulated with multiple complex environments, we demonstrate that the proposed method effectively ut…
Carnegie Mellon University（卡内基梅隆大学）；Technische Universität München（慕尼黑工业大学）；Nanyang Technological University（南洋理工大学）；Aarhus University（奥胡斯大学）；University of Washington（华盛顿大学）
Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is demand for an autonomous cinematographer that can reason about both geometry and scene context in real-time. Existing approaches do not address all aspects of this problem; they either require high-precision motion-capture systems or GPS tags to localize targets, rely on prior…
We present the dynamic model and control of DoubleBee, a novel hybrid aerial-ground vehicle consisting of two propellers mounted on tilting servo motors and two motor-driven wheels. DoubleBee exploits the high energy efficiency of a bicopter configuration in aerial mode, and enjoys the low power consumption of a two-wheel self-balancing robot on the ground. Furthermore, the propeller thrusts act as additional control inputs on the ground, enabling a novel decoupled control scheme where the attitude of the robot is controlled using thrusts and the translational motion is realized using wheels. …
Universidad Politécnica de Madrid（马德里理工大学）；Centre for Automation and Robotics (CAR), Universidad Politécnica de Madrid (UPM-CSIC)（马德里理工大学自动化与机器人中心（UPM-CSIC））；University of Luxembourg（卢森堡大学）
This paper presents an omnidirectional aerial manipulation platform for robust and responsive interaction with unstructured environments, toward the goal of contact-based inspection. The fully actuated tilt-rotor aerial system is equipped with a rigidly mounted end-effector, and is able to exert a 6 degree of freedom force and torque, decoupling the system's translational and rotational dynamics, and enabling precise interaction with the environment while maintaining stability. An impedance controller with selective apparent inertia is formulated to permit compliance in certain degrees of free…
School of Electrical Engineering and Computer Science, The University of Queensland（昆士兰大学电气工程与计算机科学学院）；CSIRO Robotics, Data61（联邦科学与工业研究组织机器人学与Data61研究院）
This article establishes the Exploration-RRT algorithm: A novel general-purpose combined exploration and pathplanning algorithm, based on a multi-goal Rapidly-Exploring Random Trees (RRT) framework. Exploration-RRT (ERRT) has been specifically designed for utilization in 3D exploration missions, with partially or completely unknown and unstructured environments. The novel proposed ERRT is based on a multi-objective optimization framework and it is able to take under consideration the potential information gain, the distance travelled, and the actuation costs, along trajectories to pseudo-rando…
Texas A&M University（德克萨斯农工大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
In this paper, an autonomous tethered Unmanned Aerial Vehicle (UAV) is developed into a visual assistant in a marsupial co-robots team, collaborating with a tele-operated Unmanned Ground Vehicle (UGV) for robot operations in unstructured or confined environments. These environments pose extreme challenges to the remote tele-operator due to the lack of sufficient situational awareness, mostly caused by the unstructuredness and confinement, stationary and limited field-of-view and lack of depth perception from the robot's onboard cameras. To overcome these problems, a secondary tele-operated rob…
Autonomous Systems Lab, ETH Zürich（苏黎世联邦理工学院自主系统实验室）
In order to enable Micro-Aerial Vehicles (MAVs) to assist in complex, unknown, unstructured environments, they must be able to navigate with guaranteed safety, even when faced with a cluttered environment they have no prior knowledge of. While trajectory optimization-based local planners have been shown to perform well in these cases, prior work either does not address how to deal with local minima in the optimization problem, or solves it by using an optimistic global planner. We present a conservative trajectory optimization-based local planner, coupled with a local exploration strategy that…
A novel decentralised trajectory generation algorithm for Multi Agent systems is presented. Multi-robot systems have the capacity to transform lives in a variety of fields. But, trajectory generation for multi-robot systems is still in its nascent stage and limited to heavily controlled environments. To overcome that, an online trajectory optimization algorithm that generates collision-free trajectories for robots, when given initial state and desired end pose, is proposed. It utilizes a simple method for obstacle detection, local shape based maps for obstacles and communication of robots' cur…
Evidence-Based Landing Site Selection and Vison-Based Landing for UAVs in Unstructured Environments
Aerospace Research Centre, National Research Council Canada（加拿大国家研究委员会航空航天研究中心）；Department of Mechanical and Industrial Engineering, Toronto Metropolitan University（多伦多 Metropolitan 大学机械与工业工程系）
This paper develops an autonomous tethered aerial visual assistant for robot operations in unstructured or confined environments. Robotic tele-operation in remote environments is difficult due to lack of sufficient situational awareness, mostly caused by the stationary and limited field-of-view and lack of depth perception from the robot's onboard camera. The emerging state of the practice is to use two robots, a primary and a secondary that acts as a visual assistant to overcome the perceptual limitations of the onboard sensors by providing an external viewpoint. However, problems exist when …
University of São Paulo（圣保罗大学）；Carnegie Mellon University（卡内基梅隆大学）
Aerial cinematography is significantly expanding the capabilities of film-makers. Recent progress in autonomous unmanned aerial vehicles (UAVs) has further increased the potential impact of aerial cameras, with systems that can safely track actors in unstructured cluttered environments. Professional productions, however, require the use of multiple cameras simultaneously to record different viewpoints of the same scene, which are edited into the final footage either in real time or in post-production. Such extreme motion coordination is particularly hard for unscripted action scenes, which are…
National Robotics Engineering Center, the Robotics Institute of Carnegie Mellon University（卡内基梅隆大学机器人研究所国家机器人工程中心）
A robotic system of multiple unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) has the potential for advancing autonomous object geolocation performance. Much research has focused on algorithmic improvements on individual components, such as navigation, motion planning, and perception. In this paper, we present a UGV-UAV object detection and geolocation system, which performs perception, navigation, and planning autonomously in real scale in unstructured environment. We designed novel sensor pods equipped with multispectral (visible, near-infrared, thermal), high resolution (…
Model predictive control (MPC) is an effective method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters. However, application of MPC can be computationally demanding, and typically requires estimating the state of the system, which can be challenging in complex, unstructured environments. Reinforcement learning can in principle forego the need for explicit state estimation and acquire a policy that directly maps sensor readings to actions, but is difficult to apply to unstable systems that are liable to fail catastrophically during training before an …
The paper proposes a reliable and robust planning solution to the long range robotic navigation problem in extremely cluttered environments. A two-layer planning architecture is proposed that leverages both the environment map and the direct depth sensor information to ensure maximal information gain out of the onboard sensors. A frontier-based pose sampling technique is used with a fast marching cost-to-go calculation to select a goal pose and plan a path to maximize robot exploration rate. An artificial potential function approach, relying on direct depth measurements, enables the robot to f…
Czech Technical University in Prague（捷克布拉格技术大学）；Brno University of Technology（布尔诺理工大学）
This paper presents a self-contained system for the robust utilization of aerial robots in the autonomous exploration of cave environments to help human explorers, first responders, and speleologists. The proposed system is generally applicable to an arbitrary exploration task within an unknown and unstructured subterranean environment and interconnects crucial robotic subsystems to provide full autonomy of the robots. Such subsystems primarily include mapping, path and trajectory planning, localization, control, and decision making. Due to the diversity, complexity, and structural uncertainty…
A Soft-Bodied Aerial Robot for Collision Resilience and Contact-Reactive Perching
Polytechnic School, Ira A. Fulton Schools of Engineering, Arizona State University（亚利桑那州立大学艾拉·A·富尔顿工程学院理工学院）；Imperial College London（伦敦帝国理工学院）；Toyota InfoTech R&D Labs（丰田信息技术研发实验室）；Hellenic Mediterranean University（希腊地中海大学）
German Aerospace Center (DLR)（德国航空航天中心（DLR））；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT））；University of Twente (UT)（特文特大学（UT））；Technical University of Munich (TUM)（慕尼黑工业大学（TUM））
This article presents a novel telepresence system for advancing aerial manipulation in dynamic and unstructured environments. The proposed system not only features a haptic device, but also a virtual reality (VR) interface that provides real-time 3D displays of the robot's workspace as well as a haptic guidance to its remotely located operator. To realize this, multiple sensors namely a LiDAR, cameras and IMUs are utilized. For processing of the acquired sensory data, pose estimation pipelines are devised for industrial objects of both known and unknown geometries. We further propose an active…
The recent availability of commercial-off-the-shelf (COTS) legged robot platforms have opened up new opportunities in deploying legged systems into different scenarios. While the main advantage of legged robots is their ability to traverse unstructured terrain, there are still large gaps between what robot platforms can achieve and their animal counterparts. Therefore, when deploying as part of a heterogeneous robot team of different platforms, it is beneficial to understand the different scenarios where a legged platform would perform better than a wheeled, tracked or aerial platform. Two COT…
Meridian: Metric-Semantic Primitive Matching for Cross-View Geo-Localization Beyond Urban Environments
Massachusetts Institute of Technology（麻省理工学院）；GRASP Laboratory, University of Pennsylvania（宾夕法尼亚大学GRASP实验室）；U.S. Army Combat Capabilities Development Command, Army Research Laboratory（美国陆军战斗能力发展指挥部，陆军研究实验室）
Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Spatialtemporal AI（时空AI）；The Hong Kong University of Science and Technology（香港科技大学）；Department of Eletronic Engineering, Tsinghua University（清华大学电子工程系）；Beijing Innovation Center of Humanoid Robotics Co., Ltd.（北京人形机器人创新中心有限公司）
