Reactive collision avoidance is essential for agile robots navigating complex and dynamic environments, enabling real-time obstacle response. However, this task is inherently challenging because it requires a tight integration of perception, planning, and control, which traditional methods often handle separately, resulting in compounded errors and delays. This paper introduces a novel approach that unifies these tasks into a single reactive framework using solely onboard sensing and computing. Our method combines nonlinear model predictive control with adaptive control barrier functions, directly linking perception-driven constraints to real-time planning and control. Constraints are determined by using a neural network to refine noisy RGB-D data, enhancing depth accuracy, and selecting points with the minimum time-to-collision to prioritize the most immediate threats. To maintain a balance between safety and agility, a heuristic dynamically adjusts the optimization process, preventing overconstraints in real time. Extensive experiments with an agile quadrotor demonstrate effective collision avoidance across diverse indoor and outdoor environments, without requiring environment-specific tuning or explicit mapping.
We present a novel, decentralized collision avoidance algorithm for navigating a swarm of quadrotors in dense environments populated with static and dynamic obstacles. Our algorithm relies on the concept of Optimal Reciprocal CollisionAvoidance (ORCA) and utilizes a flatness-based Model Predictive Control (MPC) to generate local collision-free trajectories for each quadrotor. We feedforward linearize the non-linear dynamics of the quadrotor and subsequently use this linearized model in our MPC framework. Our method is downwash conscious and computes safe trajectories that avoid quadrotors from…
Robotics Institute at Carnegie Mellon University（卡内基梅隆大学机器人研究所）
We present a multirotor architecture capable of aggressive autonomous flight and collision-free teleoperation in unstructured, GPS-denied environments. The proposed system enables aggressive and safe autonomous flight around clutter by integrating recent advancements in visual-inertial state estimation and teleoperation. Our teleoperation framework maps user inputs onto smooth and dynamically feasible motion primitives. Collision-free trajectories are ensured by querying a locally consistent map that is incrementally constructed from forward-facing depth observations. Our system enables a non-…
Micro aerial vehicles (MAVs), are frequently used for exploration, examination, and surveillance during search and rescue missions. Manually piloting these robots under stressful conditions provokes pilot errors and can result in crashes with disastrous consequences. Also, during fully autonomous flight, planned high-level trajectories can be erroneous and steer the robot into obstacles. In this work, we propose an approach to efficiently compute smooth, time-optimal trajectories MAVs that avoid obstacles. Our method first computes a trajectory from the start to an arbitrary target state, incl…
We present decentralized collision avoidance algorithms for quadrotor swarms operating under uncertain state estimation. Our approach exploits the differential flatness property and feedforward linearization to approximate the quadrotor dynamics and performs reciprocal collision avoidance. We account for the uncertainty in position and velocity by formulating the collision constraints as chance constraints, which describe a set of velocities that avoid collisions with a specified confidence level. We present two different methods for formulating and solving the chance constraints: our first me…
Robotics Research Center, IIIT Hyderabad（国际信息技术研究所海得拉巴分校机器人研究中心）；University of Tartu（塔尔图大学）
Many commodity sensors that measure the robot and dynamic obstacle's state have non-Gaussian noise characteristics. Yet, many current approaches treat the underlying-uncertainty in motion and perception as Gaussian, primarily to ensure computational tractability. On the other hand, existing planners working with non-Gaussian uncertainty do not shed light on leveraging distributional characteristics of motion and perception noise, such as bias for efficient collision avoidance. This paper fills this gap by interpreting reactive collision avoidance as a distribution matching problem between the …
EROAS: 3D Efficient Reactive Obstacle Avoidance System for Autonomous Underwater Vehicles using 2.5D Forward-Looking Sonar
Department of Aerospace Eng.（航空航天工程系）；Indian Institute of Science（印度科学研究所）；Department of Electrical and Electronics Eng.（电子与电气工程系）；Birla Institute of Technology and Science（比拉理工学院）；Department of Computer Science and Eng.（计算机科学与工程系）；Shiv Nadar University（施瓦纳大学）
Swiss Federal School of Technology in Lausanne - EPFL（洛桑联邦理工学院）；Massachusetts Institute of Technology（麻省理工学院）
Humans are remarkable at navigating and moving through dynamic and complex spaces, such as crowded streets. For robots to do the same, it is crucial that they are endowed with highly reactive obstacle avoidance robust to partial and poor sensing. We address the issue of enabling obstacle avoidance based on sparse and asynchronous perception. The proposed control scheme combines a high-level input command provided by either a planner or a human operator with fast reactive obstacle avoidance. The sampling-based sensor data can be combined with an analytical reconstruction of the obstacles for re…
Optical flow is believed to play an important role in the agile flight of birds and insects. Even though it is a very simple concept, it is rarely used in computer vision for collision avoidance. This work implements a neural network based collision avoidance which was deployed and evaluated on a solely for this purpose refitted car.
A Collision-Free Sway Damping Model Predictive Controller for Safe and Reactive Forestry Crane Navigation
Automation & Control Institute (ACIN), TU Wien（自动化与控制研究所（ACIN），维也纳技术大学）；Center for Vision, Automation & Control, AIT Austrian Institute of Technology GmbH（视觉、自动化与控制中心，奥地利技术研究所）
There are many situations for which an unmanned ground vehicle has to work with only partial observability of the environment. Therefore, a feasible nonholonomic obstacle avoidance and target tracking action must be generated immediately based on the real-time perceptual information. This paper presents a robust approach to integrating VPH+ (enhanced vector polar histogram) and MPC (model predictive control). VPH+ is applied to calculate the desired direction for its environment perception ability and computational efficiency, while MPC is explored to perform a constrained model-predictive tra…
Åbo Akademi University（奥博学术大学）；Nokia Bell Labs（诺基亚贝尔实验室）
We study the training performance of ROS local planners based on Reinforcement Learning (RL), and the trajectories they produce on real-world robots. We show that recent enhancements to the Soft Actor Critic (SAC) algorithm such as RAD and DrQ achieve almost perfect training after only 10000 episodes. We also observe that on real-world robots the resulting SACPlanner is more reactive to obstacles than traditional ROS local planners such as DWA.
Safe Expeditious Whole-Body Control of Mobile Manipulators for Collision Avoidance
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Advanced Manufacturing, Sun Yat-Sen University, Sun Yat-Sen University（中山大学先进制造学院）；School of Electrical and Automation Engineering, Hefei University of Technology（合肥工业大学电气与自动化工程学院）
This work combines control barrier functions (CBFs) with a whole-body controller to enable self-collision avoidance for the MIT Humanoid. Existing reactive controllers for self-collision avoidance cannot guarantee collision-free trajectories as they do not leverage the robot's full dynamics, thus compromising kinematic feasibility. In comparison, the proposed CBF-WBC controller can reason about the robot's underactuated dynamics in real-time to guarantee collision-free motions. The effectiveness of this approach is validated in simulation. First, a simple hand-reaching experiment shows that th…
This paper studies the collision avoidance problem for autonomous multiple fixedwing UAVs in the complex integrated airspace. By studying and combining the online path planning method, the distributed model predictive control algorithm, and the geometric reactive control approach, a three-layered collision avoidance system integrating conflict detection and resolution procedures is developed for multiple fixed-wing UAVs modeled by unicycle kinematics subject to input constraints. The effectiveness of the proposed methodology is evaluated and validated via test results of comparative simulation…
This paper presents a reactive planning system that enriches the topological representation of an environment with a tightly integrated semantic representation, achieved by incorporating and exploiting advances in deep perceptual learning and probabilistic semantic reasoning. Our architecture combines object detection with semantic SLAM, affording robust, reactive logical as well as geometric planning in unexplored environments. Moreover, by incorporating a human mesh estimation algorithm, our system is capable of reacting and responding in real time to semantically labeled human motions and g…
In this article we propose a reactive constrained navigation scheme, with embedded obstacles avoidance for an Unmanned Aerial Vehicle (UAV), for enabling navigation in obstacle-dense environments. The proposed navigation architecture is based on Nonlinear Model Predictive Control (NMPC), and utilizes an on-board 2D LiDAR to detect obstacles and translate online the key geometric information of the environment into parametric constraints for the NMPC that constrain the available position-space for the UAV. This article focuses also on the real-world implementation and experimental validation of…
The American University in Cairo（开罗美国大学）；Valeo Schalter und Sensoren GmbH（法雷奥开关与传感器有限公司）
Collision avoidance systems play a vital role in reducing the number of vehicle accidents and saving human lives. This paper extends the previous work using evolutionary neural networks for reactive collision avoidance. We are proposing a new method we have called symmetric neural networks. The method improves the model's performance by enforcing constraints between the network weights which reduces the model optimization search space and hence, learns more accurate control of the vehicle steering for improved maneuvering. The training and validation processes are carried out using a simulatio…
MARS-FTCP: Robust Fault-Tolerant Control and Agile Trajectory Planning for Modular Aerial Robot Systems
Department of Electrical and Computer Engineering, National University of Singapore（电子与计算机工程系，国立新加坡大学）；Engineering Design and Innovation Centre, National University of Singapore（工程设计与创新中心，国立新加坡大学）
FZI Research Center for Information Technology（FZI信息技术研究中心）
We present a biologically inspired approach for path planning with dynamic obstacle avoidance. Path planning is performed in a condensed configuration space of a robot generated by self-organizing neural networks (SONN). The robot itself and static as well as dynamic obstacles are mapped from the Cartesian task space into the configuration space by precomputed kinematics. The condensed space represents a cognitive map of the environment, which is inspired by place cells and the concept of cognitive maps in mammalian brains. The generation of training data as well as the evaluation were perform…
Dorabot Inc.（Dorabot公司）；City University of Hong Kong（香港城市大学）；Fuzhou University（福州大学）
High-speed, low-latency obstacle avoidance that is insensitive to sensor noise is essential for enabling multiple decentralized robots to function reliably in cluttered and dynamic environments. While other distributed multi-agent collision avoidance systems exist, these systems require online geometric optimization where tedious parameter tuning and perfect sensing are necessary. We present a novel end-to-end framework to generate reactive collision avoidance policy for efficient distributed multi-agent navigation. Our method formulates an agent's navigation strategy as a deep neural network …
StratMamba: Strategic and Reactive Stream Partitioning for Path-Efficient LiDAR-Based Obstacle Avoidance
The Pennsylvania State University（宾夕法尼亚州立大学）；AlphaZ, Inc.（阿尔法兹公司）；Georgia Institute of Technology（佐治亚理工学院）；Massachusetts Institute of Technology（麻省理工学院）
Safe and efficient collaboration among multiple robots in unstructured environments is increasingly critical in the era of Industry 4.0. However, achieving robust and autonomous collaboration among humans and other robots requires modern robotic systems to have effective proximity perception and reactive obstacle avoidance. In this paper, we propose a novel methodology for reactive whole-body obstacle avoidance that ensures conflict-free robot-robot interactions even in dynamic environment. Unlike existing approaches based on Jacobian-type, sampling based or geometric techniques, our methodolo…
University of Pennsylvania（宾夕法尼亚大学）；University of California, Berkeley（加州大学伯克利分校）；Oregon State University（俄勒冈州立大学）
With increasing urban population, there is global interest in Urban Air Mobility (UAM), where hundreds of autonomous Unmanned Aircraft Systems (UAS) execute missions in the airspace above cities. Unlike traditional human-in-the-loop air traffic management, UAM requires decentralized autonomous approaches that scale for an order of magnitude higher aircraft densities and are applicable to urban settings. We present Learning-to-Fly (L2F), a decentralized on-demand airborne collision avoidance framework for multiple UAS that allows them to independently plan and safely execute missions with spati…
Munich School of Robotics and Machine Intelligence（慕尼黑机器人与机器智能学院）；Technical University of Munich（慕尼黑工业大学）
In this paper, we address the problem of vision-based obstacle avoidance for robotic manipulators. This topic poses challenges for both perception and motion generation. While most work in the field aims at improving one of those aspects, we provide a unified framework for approaching this problem. The main goal of this framework is to connect perception and motion by identifying the relationship between the visual input and the corresponding motion representation. To this end, we propose a method for learning reactive obstacle avoidance policies. We evaluate our method on goal-reaching tasks …
Bristol Robotics Laboratory, The University of Bristol（布里斯托尔机器人实验室，布里斯托尔大学）；Visual Information Laboratory, The University of Bristol（视觉信息实验室，布里斯托尔大学）；Department of Electrical and Electronic Engineering, The University of Manchester（曼彻斯特大学电气与电子工程系）；The Department of Computer Science and Engineering & The Department of Ophthalmology, The University of California, San Diego（加利福尼亚大学圣地亚哥分校计算机科学与工程系及眼科系）；Hangzhou ATG Intelligent Technology Co. Ltd.（杭州ATG智能科技有限公司）；Amazon（亚马逊公司）；The University of California, San Diego（加利福尼亚大学圣地亚哥分校）；The University of Manchester（曼彻斯特大学）；The University of Bristol（布里斯托尔大学）；Amazon, Seattle, USA（亚马逊公司（西雅图））；The Department of Ophthalmology, The University of California, San Diego（加利福尼亚大学圣地亚哥分校眼科系）
This paper presents an agile reactive navigation strategy for driving a non-holonomic ground vehicle around a preset course of gates in a cluttered environment using a low-cost processor array sensor. This enables machine vision tasks to be performed directly upon the sensor's image plane, rather than using a separate general-purpose computer. We demonstrate a small ground vehicle running through or avoiding multiple gates at high speed using minimal computational resources. To achieve this, target tracking algorithms are developed for the Pixel Processing Array and captured images are then pr…
This paper proposes a framework for 3D obstacle avoidance in the presence of partial observability of environment obstacles. The method focuses on the utility of the Artificial Potential Function (APF) controller in a practical setting where noisy and incomplete information about the proximity is inevitable. We propose a Particle Filter (PF) approach to estimate potential obstacle locations in an input depth image stream. The probable candidates are then used to generate an action that maneuvers the robot towards the negative gradient of potential at each time instant. Rigorous experimental va…
Multiple multirotor Micro Aerial Vehicles sharing the same airspace require a reliable and robust collision avoidance technique. In this paper we address the problem of multi-MAV reactive collision avoidance. A model-based controller is employed to achieve simultaneously reference trajectory tracking and collision avoidance. Moreover, we also account for the uncertainty of the state estimator and the other agents position and velocity uncertainties to achieve a higher degree of robustness. The proposed approach is decentralized, does not require collision-free reference trajectory and accounts…
Local Path Planning and Obstacle Avoidance for an Omnicopter Platform
Faculty of Aerospace Engineering, Delft University of Technology（代尔夫特理工大学航空航天工程学院）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Center for Artificial Intelligence and Robotics, New York University Abu Dhabi（纽约大学阿布扎比分校人工智能与机器人中心）
Hybrid Task and Motion Planning with Reactive Collision Handling for Multi-Robot Disassembly of Complex Products: Application to EV Batteries
Extreme Robotics Laboratory, School of Metallurgy and Materials, University of Birmingham（极端机器人实验室，冶金与材料学院，伯明翰大学）；Department of Applied Artificial Intelligence and Robotics, Aston University（应用人工智能与机器人学院，阿斯顿大学）
Edinburgh Centre for Robotics（爱丁堡机器人中心）；University of Edinburgh（爱丁堡大学）；Heriot-Watt University（赫瑞-瓦特大学）
Unforeseen events are frequent in the real-world environments where robots are expected to assist, raising the need for fast replanning of the policy in execution to guarantee the system and environment safety. Inspired by human behavioural studies of obstacle avoidance and route selection, this paper presents a hierarchical framework which generates reactive yet bounded obstacle avoidance behaviours through a multi-layered analysis. The framework leverages the strengths of learning techniques and the versatility of dynamic movement primitives to efficiently unify perception, decision, and act…
University of Pennsylvania（宾夕法尼亚大学）；Massachusetts Institute of Technology（麻省理工学院）
This paper considers the problem of completing assemblies of passive objects in nonconvex environments, cluttered with convex obstacles of unknown position, shape and size that satisfy a specific separation assumption. A differential drive robot equipped with a gripper and a LIDAR sensor, capable of perceiving its environment only locally, is used to position the passive objects in a desired configuration. The method combines the virtues of a deliberative planner generating high-level, symbolic commands, with the formal guarantees of convergence and obstacle avoidance of a reactive planner tha…
Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）
With the advancement of robotics, machine learning, and machine perception, increasingly more robots will enter human environments to assist with daily tasks. However, dynamically-changing human environments requires reactive motion plans. Reactivity can be accomplished through replanning, e.g. model-predictive control, or through a reactive feedback policy that modifies on-going behavior in response to sensory events. In this paper, we investigate how to use machine learning to add reactivity to a previously learned nominal skilled behavior. We approach this by learning a reactive modificatio…
