This paper presents PANTHER, a real-time perception-aware (PA) trajectory planner for multirotor-UAVs (Unmanned Aerial Vehicles) in dynamic environments. PANTHER plans trajectories that avoid dynamic obstacles while also keeping them in the sensor field of view (FOV) and minimizing the blur to aid in object tracking. The rotation and translation of the UAV are jointly optimized, which allows PANTHER to fully exploit the differential flatness of multirotors to maximize the PA objective. Real-time performance is achieved by implicitly imposing the underactuated dynamics of the UAV through the Hopf fibration. PANTHER is able to keep the obstacles inside the FOV 7.9 and 1.5 times more than non-PA approaches and PA approaches that decouple translation and yaw, respectively. The projected velocity (and hence the blur) is reduced by 18% and 34%, respectively. This leads to average success rates three times larger than state-of-the-art approaches in multi-obstacle avoidance scenarios. The MINVO basis is used to impose low-conservative collision avoidance constraints in position and velocity space. Finally, extensive hardware experiments in unknown dynamic environments with all the computation running onboard are presented, with velocities of up to 5.8 m/s, and with relative velocities (with respect to the obstacles) of up to 6.3 m/s. The only sensors used are an IMU, a forward-facing depth camera, and a downward-facing monocular camera.
This paper presents Deep-PANTHER, a learning-based perception-aware trajectory planner for unmanned aerial vehicles (UAVs) in dynamic environments. Given the current state of the UAV, and the predicted trajectory and size of the obstacle, Deep-PANTHER generates multiple trajectories to avoid a dynamic obstacle while simultaneously maximizing its presence in the field of view (FOV) of the onboard camera. To obtain a computationally tractable real-time solution, imitation learning is leveraged to train a Deep-PANTHER policy using demonstrations provided by a multimodal optimization-based expert.…
Connectivity technology has shown great potentials in improving the safety and efficiency of transportation systems by providing information beyond the perception and prediction capabilities of individual vehicles. However, it is expected that human-driven and autonomous vehicles, and connected and non-connected vehicles need to share the transportation network during the transition period to fully connected and automated transportation systems. Such mixed traffic scenarios significantly increase the complexity in analyzing system behavior and quantifying uncertainty for highly interactive sce…
Tag-based visual-inertial localization is a lightweight method for enabling autonomous data collection missions of low-cost unmanned aerial vehicles (UAVs) in indoor construction environments. However, finding the optimal tag configuration (i.e., number, size, and location) on dynamic construction sites remains challenging. This paper proposes a perception-aware genetic algorithm-based tag placement planner (PGA-TaPP) to determine the optimal tag configuration using 4D-BIM, considering the project progress, safety requirements, and UAV's localizability. The proposed method provides a 4D plan f…
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）；GraphCore；Massachusetts Institute of Technology（麻省理工学院）
The motion planners used in self-driving vehicles need to generate trajectories that are safe, comfortable, and obey the traffic rules. This is usually achieved by two modules: behavior planner, which handles high-level decisions and produces a coarse trajectory, and trajectory planner that generates a smooth, feasible trajectory for the duration of the planning horizon. These planners, however, are typically developed separately, and changes in the behavior planner might affect the trajectory planner in unexpected ways. Furthermore, the final trajectory outputted by the trajectory planner mig…
The use of machine learning in the self-driving industry has boosted a number of recent advancements. In particular, the usage of large deep learning models in the perception and prediction stack have proved quite successful, but there still lacks significant literature on the use of machine learning in the planning stack. The current state of the art in the planning stack often relies on fast constrained optimization or rule-based approaches. Both of these techniques fail to address a significant number of fundamental problems that would allow the vehicle to operate more similarly to that of …
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；Institute of Robotics and Intelligent Manufacturing (IRIM), The Chinese University of Hong Kong (CUHK)（香港中文大学机器人与智能制造研究院）；George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院乔治·W·伍德拉夫机械工程学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
Dynamic quadrupedal locomotion over rough terrains reveals remarkable progress over the last few decades. Small-scale quadruped robots are adequately flexible and adaptable to traverse uneven terrains along sagittal direction, such as slopes and stairs. To accomplish autonomous locomotion navigation in complex environments, spinning is a fundamental yet indispensable functionality for legged robots. However, spinning behaviors of quadruped robots on uneven terrain often exhibit position drifts. Motivated by this problem, this study presents an algorithmic method to enable accurate spinning mot…
Technical University of Munich（慕尼黑工业大学）；Institute for Automotive Engineering, Munich Institute of Robotics and Machine Intelligence, School of Engineering and Design（汽车工程研究所，慕尼黑机器人与机器智能研究所，工程与设计学院）
PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
Technical University of Munich（慕尼黑工业大学）；TUM School of Engineering and Design（慕尼黑工业大学工程与设计学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；University College London（伦敦大学学院）
Horizon Robotics；University of Hong Kong（香港大学）；University of the Chinese Academy of Sciences（中国科学院大学）；Nanjing University（南京大学）；Beijing Jiaotong University（北京交通大学）
Visual inertial odometry (VIO) is widely used for the state estimation of multicopters, but it may function poorly in environments with few visual features or in overly aggressive flights. In this work, we propose a perception-aware collision avoidance trajectory planner for multicopters, that may be used with any feature-based VIO algorithm. Our approach is able to fly the vehicle to a goal position at fast speed, avoiding obstacles in an unknown stationary environment while achieving good VIO state estimation accuracy. The proposed planner samples a group of minimum jerk trajectories and fin…
Visual navigation has been widely used for state estimation of micro aerial vehicles (MAVs). For stable visual navigation, MAVs should generate perception-aware paths which guarantee enough visible landmarks. Many previous works on perception-aware path planning focused on sampling-based planners. However, they may suffer from sample inefficiency, which leads to computational burden for finding a global optimal path. To address this issue, we suggest a perception-aware path planner which utilizes topological information of environments. Since the topological class of a path and visible landmar…
Active Robotic Perception for Disease Detection and Mapping in Apple Trees
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Department of Plant Pathology, Virginia Polytechnic Institute and State University（弗吉尼亚理工大学植物病理学系）
This paper proposes a perception and path planning pipeline for autonomous racing in an unknown bounded course. The pipeline was initially created for the 2021 evGrandPrix autonomous division and was further improved for the 2022 event, both of which resulting in first place finishes. Using a simple LiDAR-based perception pipeline feeding into an occupancy grid based expansion algorithm, we determine a goal point to drive. This pipeline successfully achieved reliable and consistent laps in addition with occupancy grid algorithm to know the ways around a cone-defined track with an averaging spe…
Look as You Leap: Planning Simultaneous Motion and Perception for High-DOF Robots
Department of Computer Science, Rice University（计算机科学系，里士大学）；Ken Kennedy Institute, Rice University（肯尼迪研究所，里士大学）；Houston Methodist Academic Institute（休斯顿 Methodist 学术研究院）
Department of Aeronautics and Astronautics, Massachusetts Institute of Technology（航空与航天系，麻省理工学院）；Institute for Research in Technology, ICAI School of Engineering, Comillas Pontifical University (Spain)（技术研究所，ICAI工程学院，Comillas天主教大学（西班牙））
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Korea Atomic Energy Research Institute（韩国原子能研究院）
Although bipedal locomotion provides the ability to traverse unstructured environments, it requires careful planning and control to safely walk across without falling. This poses an integrated challenge for the robot to perceive, plan, and control its movements, especially with dynamic motions where the robot may have to adapt its swing-leg trajectory onthe-fly in order to safely place its foot on the uneven terrain. In this paper we present an efficient geometric footstep planner and the corresponding walking controller that enables a humanoid robot to dynamically walk across uneven terrain a…
Learning from demonstration for motion planning is an ongoing research topic. In this paper we present a model that is able to learn the complex mapping from raw 2D-laser range findings and a target position to the required steering commands for the robot. To our best knowledge, this work presents the first approach that learns a target-oriented end-to-end navigation model for a robotic platform. The supervised model training is based on expert demonstrations generated in simulation with an existing motion planner. We demonstrate that the learned navigation model is directly transferable to pr…
FLAP: FOV-Constrained Active Perception Planning for Prior-Map-Free 3D Navigation
The State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院工业控制技术国家重点实验室）；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）；Huzhou Key Laboratory of Autonomous System（湖州市自动驾驶系统重点实验室）；Shanghai Institute of Special Equipment Inspection and Technical Research Co., Ltd（上海市特种设备监督检验技术研究院有限公司）
Institute for Robotics and Intelligent Machines（机器人与智能机械研究所）；Georgia Institute of Technology（佐治亚理工学院）；School of Electrical and Computer Engineering（电气与计算机工程学院）
Korea Advanced Institute of Science and Technology（韩国科学技术院）
As the aging population grows at a rapid rate, there is an ever growing need for service robot platforms that can provide daily assistance at practical speed with reliable performance. In order to assist with daily tasks such as fetching a beverage, a service robot must be able to perceive its environment and generate corresponding motion trajectories. This becomes a challenging and computationally complex problem when the environment is unknown and thus the path planner must sample numerous trajectories that often are sub-optimal, extending the execution time. To address this issue, we propos…
Planning in unstructured environments is challenging -- it relies on sensing, perception, scene reconstruction, and reasoning about various uncertainties. We propose DeepSemanticHPPC, a novel uncertainty-aware hypothesis-based planner for unstructured environments. Our algorithmic pipeline consists of: a deep Bayesian neural network which segments surfaces with uncertainty estimates; a flexible point cloud scene representation; a next-best-view planner which minimizes the uncertainty of scene semantics using sparse visual measurements; and a hypothesis-based path planner that proposes multiple…
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Motion planners take uncertain information about the environment as an input. The environment information is often quite noisy and has a tendency to contain false positive object detection. State-of-the-art motion planners consider all objects alike, thus producing overcautious behavior. In this paper we present a planning approach that considers alternative maneuvers in a combined fashion and plans a motion that is formed by the probabilities of those alternatives. The proposed planner can smoothly react to objects with low existence probability while remaining collision-free in case their ex…
