DikpolaSat Mission: Improvement of Space Flight Performance and Optimal Control Using Trained Deep Neural Network -- Trajectory Controller for Space Objects Collision Avoidance
Tod’Aérs - Transparent Organization for Aeronautics and Space Research（Tod'Aérs - 航空航天研究透明组织）；Space Generation Advisory Council（太空世代咨询委员会）；Moscow Aviation Institute（莫斯科航空学院）；Univerité Sultan Moulay Slimane（苏丹穆莱·斯利曼大学）
This paper introduced the space mission DikpolaSat Mission, how this research fits into the mission, and the importance of having a trained DNN model instead of the usual GN&C functionality. This paper shows how the controller demonstration is carried out by having the spacecraft follow a desired path, specified in the referenced model. Increases can be made by examining the route used to construct a DNN and understanding the effects of various activating functions on system efficiency. The obstacle avoidance algorithm is built into the control features to respond spontaneously using inputs from the neural network for collision avoidance while optimizing the modified trajectory. The action of a neural network to control the adaptive nature of the nonlinear mechanisms in the controller will make the control system capable of handling multiple nonlinear events and also uncertainties that have not been induced in the control algorithm. Multiple algorithms for optimizing flight controls and fuel consumption can be implemented using knowledge of flight dynamics in trajectory and also in the event of obstacle avoidance. This paper also explains how a DNN can learn to control the flight path and make the system more reliable with each launch, thereby improving the chances of predicting collisions of space objects. The data released from this research is used to design more advanced DNN model capable of predicting other orbital events as well.
This paper presents a robust computationally efficient real-time collision avoidance algorithm for Unmanned Aerial Vehicle (UAV), namely Memory-based Wall Following-Artificial Potential Field (MWF-APF) method. The new algorithm switches between Wall-Following Method (WFM) and Artificial Potential Field method (APF) with improved situation awareness capability. Historical trajectory is taken into account to avoid repetitive wrong decision. Furthermore, it can be effectively applied to platform with low computing capability. As an example, a quad-rotor equipped with limited number of Time-of-Fli…
Mapless Collision-Free Flight via MPC using Dual KD-Trees in Cluttered Environments
Shanghai Key Laboratory of Navigation and Location-Based Services, School of Sensing Science and Engineering, Shanghai Jiao Tong University（上海导航与位置服务重点实验室，传感科学与工程学院，上海交通大学）
Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE（弗劳恩霍夫通信、信息处理与工效学研究所FKIE）；University of Bonn（波恩大学）
Safe navigation of Micro Aerial Vehicles (MAVs) requires not only obstacle-free flight paths according to a static environment map, but also the perception of and reaction to previously unknown and dynamic objects. This implies that the onboard sensors cover the current flight direction. Due to the limited payload of MAVs, full sensor coverage of the environment has to be traded off with flight time. Thus, often only a part of the environment is covered. We present a combined allocentric complete planning and trajectory optimization approach taking these sensor visibility constraints into acco…
This paper presents a new trajectory planning method for multiple quadrotors in obstacle-dense environments. We suggest a relative safe flight corridor (RSFC) to model safe region between a pair of agents, and it is used to generate linear constraints for inter-collision avoidance by utilizing the convex hull property of relative Bernstein polynomial. Our approach employs a graph-based multi-agent pathfinding algorithm to generate an initial trajectory, which is used to construct a safe flight corridor (SFC) and RSFC. We express the trajectory as a piecewise Bernstein polynomial and formulate …
Zhejiang University（浙江大学）；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）
For aerial swarms, navigation in a prescribed formation is widely practiced in various scenarios. However, the associated planning strategies typically lack the capability of avoiding obstacles in cluttered environments. To address this deficiency, we present an optimization-based method that ensures collision-free trajectory generation for formation flight. In this paper, a novel differentiable metric is proposed to quantify the overall similarity distance between formations. We then formulate this metric into an optimization framework, which achieves spatial-temporal planning using polynomia…
In recent years, unmanned aerial vehicles (UAVs) are used for numerous inspection and video capture tasks. Manually controlling UAVs in the vicinity of obstacles is challenging, however, and poses a high risk of collisions. Even for autonomous flight, global navigation planning might be too slow to react to newly perceived obstacles. Disturbances such as wind might lead to deviations from the planned trajectories. In this work, we present a fast predictive obstacle avoidance method that does not depend on higher-level localization or mapping and maintains the dynamic flight capabilities of UAV…
University of Washington（华盛顿大学）；University of California, Berkeley（加州大学伯克利分校）
Deep reinforcement learning has emerged as a promising and powerful technique for automatically acquiring control policies that can process raw sensory inputs, such as images, and perform complex behaviors. However, extending deep RL to real-world robotic tasks has proven challenging, particularly in safety-critical domains such as autonomous flight, where a trial-and-error learning process is often impractical. In this paper, we explore the following question: can we train vision-based navigation policies entirely in simulation, and then transfer them into the real world to achieve real-world…
The significant components of any successful autonomous flight system are task completion and collision avoidance. Most deep learning algorithms successfully execute these aspects under the environment and conditions they are trained. However, they fail when subjected to novel environments. This paper presents an autonomous multi-rotor flight algorithm, using Deep Reinforcement Learning augmented with Self-Attention Models, that can effectively reason when subjected to varying inputs. In addition to their reasoning ability, they are also interpretable, enabling it to be used under real-world c…
Collision avoidance is a crucial task in vision-guided autonomous navigation. Solutions based on deep reinforcement learning (DRL) has become increasingly popular. In this work, we proposed several novel agent state and reward function designs to tackle two critical issues in DRL-based navigation solutions: 1) smoothness of the trained flight trajectories; and 2) model generalization to handle unseen environments. Formulated under a DRL framework, our model relies on margin reward and smoothness constraints to ensure UAVs fly smoothly while greatly reducing the chance of collision. The propose…
Robotics Institute at Carnegie Mellon University（卡内基梅隆大学机器人研究所）
We present a multirotor architecture capable of aggressive autonomous flight and collision-free teleoperation in unstructured, GPS-denied environments. The proposed system enables aggressive and safe autonomous flight around clutter by integrating recent advancements in visual-inertial state estimation and teleoperation. Our teleoperation framework maps user inputs onto smooth and dynamically feasible motion primitives. Collision-free trajectories are ensured by querying a locally consistent map that is incrementally constructed from forward-facing depth observations. Our system enables a non-…
In this work, we propose a method to efficiently compute smooth, time-optimal trajectories for micro aerial vehicles (MAVs) evading a moving obstacle. Our approach first computes an n-dimensional trajectory from the start- to an arbitrary target state including position, velocity and acceleration. It respects input- and state-constraints and is thus dynamically feasible. The trajectory is then efficiently checked for collisions, exploiting the piecewise polynomial formulation. If collisions occur, viastates are inserted into the trajectory to circumvent the obstacle and still maintain time-opt…
Quadruped Obstacle Avoidance and Footstep Planning with Distributed Low-cost Time-of-Flight Sensors
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）；Free University of Bozen-Bolzano（博尔扎诺自由大学）；University of Edinburgh（爱丁堡大学）；University of Udine（乌迪内大学）
Optical flow is believed to play an important role in the agile flight of birds and insects. Even though it is a very simple concept, it is rarely used in computer vision for collision avoidance. This work implements a neural network based collision avoidance which was deployed and evaluated on a solely for this purpose refitted car.
A collision avoidance system based on simple digital cameras would help enable the safe integration of small UAVs into crowded, low-altitude environments. In this work, we present an obstacle avoidance system for small UAVs that uses a monocular camera with a hybrid neural network and path planner controller. The system is comprised of a vision network for estimating depth from camera images, a high-level control network, a collision prediction network, and a contingency policy. This system is evaluated on a simulated UAV navigating an obstacle course in a constrained flight pattern. Results s…
Zhejiang University（浙江大学）；Huzhou Institute of Zhejiang University（湖州学院（浙江大学湖州研究院））
In this paper, we introduce a complete system for autonomous flight of quadrotors in dynamic environments with onboard sensing. Extended from existing work, we develop an occlusion-aware dynamic perception method based on depth images, which classifies obstacles as dynamic and static. For representing generic dynamic environment, we model dynamic objects with moving ellipsoids and fuse static ones into an occupancy grid map. To achieve dynamic avoidance, we design a planning method composed of modified kinodynamic path searching and gradient-based optimization. The method leverages manually co…
High-Speed Vision-Based Flight in Clutter with Safety-Shielded Reinforcement Learning
Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院工业控制技术研究所）；Differential Robotics（微分机器人）
Omnidirectional Solid-State mmWave Radar Perception for UAV Power Line Collision Avoidance
Drone Infrastructure Inspection and Interaction Group, Section for Digital and High-Frequency Electronics, Department of Mechanical and Electrical Engineering, Faculty of Engineering, University of Southern Denmark（丹麦南部大学工程学院机械与电气工程系数字与高频电子学部无人机基础设施检查与交互小组）
AMSwarmX: Safe Swarm Coordination in CompleX Environments via Implicit Non-Convex Decomposition of the Obstacle-Free Space
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；Technical University of Munich（慕尼黑工业大学）；Vector Institute for Artificial Intelligence（向量人工智能研究所）；University of Tartu（塔尔图大学）
Massachusetts Institute of Technology（麻省理工学院）；University of California, San Diego（加利福尼亚大学圣地亚哥分校）
Safety and stability are common requirements for robotic control systems; however, designing safe, stable controllers remains difficult for nonlinear and uncertain models. We develop a model-based learning approach to synthesize robust feedback controllers with safety and stability guarantees. We take inspiration from robust convex optimization and Lyapunov theory to define robust control Lyapunov barrier functions that generalize despite model uncertainty. We demonstrate our approach in simulation on problems including car trajectory tracking, nonlinear control with obstacle avoidance, satell…
MC-Swarm: Minimal-Communication Multi-Agent Trajectory Planning and Deadlock Resolution for Quadrotor Swarm
AI Institute of Seoul National University（首尔国立大学人工智能研究所）；Carnegie Mellon University（卡内基梅隆大学）；Department of Mechanical System Design Engineering, Seoul National University of Science and Technology (SEOULTECH)（首尔科学技术大学机械系统设计工程系）
Interest in unmanned aerial system (UAS) powered solutions for 6G communication networks has grown immensely with the widespread availability of machine learning based autonomy modules and embedded graphical processing units (GPUs). While these technologies have revolutionized the possibilities of UAS solutions, designing an operable, robust autonomy framework for UAS remains a multi-faceted and difficult problem. In this work, we present our novel, modular framework for UAS autonomy, entitled MR-iFLY, and discuss how it may be extended to enable 6G swarm solutions. We begin by detailing the c…
New methodologies will be needed to ensure the airspace remains safe and efficient as traffic densities rise to accommodate new unmanned operations. This paper explores how unmanned free-flight traffic may operate in dense airspace. We develop and analyze autonomous collision avoidance systems for aircraft operating in dense airspace where traditional collision avoidance systems fail. We propose a metric for quantifying the decision burden on a collision avoidance system as well as a metric for measuring the impact of the collision avoidance system on airspace. We use deep reinforcement learni…
Enhanced Trust Region Sequential Convex Optimization for Multi-Drone Thermal Screening Trajectory Planning in Urban Environments
Institute of Automation, Chinese Academy of Science, China（中国科学院自动化研究所）；Beijing Institute of Technology, China（北京理工大学）；Institute of Automation, Chinese Academy of Sciences, China（中国科学院自动化研究所）；Tsinghua University, China（清华大学）
Obstacle avoidance is an essential topic in the field of autonomous drone research. When choosing an avoidance algorithm, many different options are available, each with their advantages and disadvantages. As there is currently no consensus on testing methods, it is quite challenging to compare the performance between algorithms. In this paper, we propose AvoidBench, a benchmarking suite which can evaluate the performance of vision-based obstacle avoidance algorithms by subjecting them to a series of tasks. Thanks to the high fidelity of multi-rotors dynamics from RotorS and virtual scenes of …
Unmanned Aerial Vehicles (UAVs) are now becoming increasingly accessible to amateur and com-mercial users alike. Several types of airspace structures are proposed in recent research, which include several structured free flight concepts. In this paper, for simplic-ity, distributed coordinating the motions of multicopters in structured airspace concepts is focused. This is formulated as a free flight problem, which includes convergence to destination lines and inter-agent collision avoidance. The destination line of each multicopter is known a priori. Further, Lyapunov-like functions are design…
Collision avoidance requires tradeoffs in planning time horizons. Depending on the planner, safety cannot always be guaranteed in uncertain environments given map updates. To mitigate situations where the planner leads the vehicle into a state of collision or the vehicle reaches a point where no trajectories are feasible, we propose a continuous collision checking algorithm. The imminent collision checking system continuously monitors vehicle safety, and plans a safe trajectory that leads the vehicle to a stop within the observed map. We test our proposed pipeline alongside a teleoperated navi…
EcoFlight: Finding Low-Energy Paths Through Obstacles for Autonomous Sensing Drones
Computer Science New Jersey Institute of Technology Newark, NJ, USA（计算机科学 新 jersey 理工学院 新ARK, NJ, USA）；Computer Engineering New York Institute of Technology New York, NY, USA（电子与计算机工程 新 york 理工学院 新 YORK, NY, USA）
UC Berkeley（加州大学伯克利分校）；Seoul National University（首尔大学）
This paper introduces a collision avoidance system for navigating a multicopter in cluttered outdoor environments based on the recent memory-less motion planner, rectangular pyramid partitioning using integrated depth sensors (RAPPIDS). The RAPPIDS motion planner generates collision-free flight trajectories at high speed with low computational cost using only the latest depth image. In this work we extend it to improve the performance of the planner by taking the following issues into account. (a) Changes in the dynamic characteristics of the multicopter that occur during flight, such as chang…
In this paper, a speed and separation monitoring (SSM) based safety controller using three time-of-flight ranging sensor arrays fastened to the robot links, is implemented. Based on the human-robot minimum distance and their relative velocities, a controller output characterized by a modulating robot operation speed is obtained. To avert self-avoidance, a self occlusion detection method is implemented using ray-casting technique to filter out the distance values associated with the robot-self and the restricted robot workspace. For validation, the robot workspace is monitored using a motion ca…
This paper proposes a perception-shared and swarm trajectory global optimal (STGO) algorithm fused UAVs formation motion planning framework aided by an active sensing system. First, the point cloud received by each UAV is fit by the gaussian mixture model (GMM) and transmitted in the swarm. Resampling from the received GMM contributes to a global map, which is used as the foundation for consensus. Second, to improve flight safety, an active sensing system is designed to plan the observation angle of each UAV considering the unknown field, overlap of the field of view (FOV), velocity direction …
Multi-Agent gatekeeper: Safe Flight Planning and Formation Control for Urban Air Mobility
Department of Electrical & Computer Engineering（电气与计算机工程系）；Department of Robotics（机器人学系）；Department of Robotics and Department of Aerospace Engineering（机器人学系和航空航天工程系）
Obstacle avoidance from monocular images is a challenging problem for robots. Though multi-view structure-from-motion could build 3D maps, it is not robust in textureless environments. Some learning based methods exploit human demonstration to predict a steering command directly from a single image. However, this method is usually biased towards certain tasks or demonstration scenarios and also biased by human understanding. In this paper, we propose a new method to predict a trajectory from images. We train our system on more diverse NYUv2 dataset. The ground truth trajectory is computed from…
DPNet: Efficient Dead-End Prediction and Avoidance for Vision-Based UAV Navigation
Huzhou Institute of Zhejiang University（浙江大学湖州研究院）；University of Electronic Science and Technology of China（电子科技大学）；Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院 cyber系统与控制研究所）；Differential Robotics Technology Company（差分机器人技术公司）
