Autonomous spacecraft relative navigation technology has been planned for and applied to many famous space missions. The development of on-board electronics systems has enabled the use of vision-based and LiDAR-based methods to achieve better performances. Meanwhile, deep learning has reached great success in different areas, especially in computer vision, which has also attracted the attention of space researchers. However, spacecraft navigation differs from ground tasks due to high reliability requirements but lack of large datasets. This survey aims to systematically investigate the current deep learning-based autonomous spacecraft relative navigation methods, focusing on concrete orbital applications such as spacecraft rendezvous and landing on small bodies or the Moon. The fundamental characteristics, primary motivations, and contributions of deep learning-based relative navigation algorithms are first summarised from three perspectives of spacecraft rendezvous, asteroid exploration, and terrain navigation. Furthermore, popular visual tracking benchmarks and their respective properties are compared and summarised. Finally, potential applications are discussed, along with expected impediments.
City, University of London（伦敦大学城市学院）；European Space Agency（欧洲空间局）
Recent interest in on-orbit servicing and Active Debris Removal (ADR) missions have driven the need for technologies to enable non-cooperative rendezvous manoeuvres. Such manoeuvres put heavy burden on the perception capabilities of a chaser spacecraft. This paper demonstrates Convolutional Neural Networks (CNNs) capable of providing an initial coarse pose estimation of a target from a passive thermal infrared camera feed. Thermal cameras offer a promising alternative to visible cameras, which struggle in low light conditions and are susceptible to overexposure. Often, thermal information on t…
We propose factor graph optimization for simultaneous planning, control, and trajectory estimation for collision-free navigation of autonomous systems in environments with moving objects. The proposed online probabilistic motion planning and trajectory estimation navigation technique generates optimal collision-free state and control trajectories for autonomous vehicles when the obstacle motion model is both unknown and known. We evaluate the utility of the algorithm to support future autonomous robotic space missions.
Georgia Institute of Technology（佐治亚理工学院）；NASA Goddard Space Flight Center（美国国家航空航天局戈达德太空飞行中心）
We propose AstroSLAM, a standalone vision-based solution for autonomous online navigation around an unknown target small celestial body. AstroSLAM is predicated on the formulation of the SLAM problem as an incrementally growing factor graph, facilitated by the use of the GTSAM library and the iSAM2 engine. By combining sensor fusion with orbital motion priors, we achieve improved performance over a baseline SLAM solution. We incorporate orbital motion constraints into the factor graph by devising a novel relative dynamics factor, which links the relative pose of the spacecraft to the problem o…
Massachusetts Institute of Technology（麻省理工学院）；Draper（德雷珀实验室）
Terrain relative navigation can improve the precision of a spacecraft's position estimate by detecting global features that act as supplementary measurements to correct for drift in the inertial navigation system. This paper presents a system that uses a convolutional neural network (CNN) and image processing methods to track the location of a simulated spacecraft with an extended Kalman filter (EKF). The CNN, called LunaNet, visually detects craters in the simulated camera frame and those detections are matched to known lunar craters in the region of the current estimated spacecraft position.…
Florida Institute of Technology（佛罗里达理工学院）；Energy Management Aerospace（能源管理航空航天公司）
Autonomous navigation and path-planning around non-cooperative space objects is an enabling technology for on-orbit servicing and space debris removal systems. The navigation task includes the determination of target object motion, the identification of target object features suitable for grasping, and the identification of collision hazards and other keep-out zones. Given this knowledge, chaser spacecraft can be guided towards capture locations without damaging the target object or without unduly the operations of a servicing target by covering up solar arrays or communication antennas. One w…
The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Jet Propulsion Laboratory（喷气推进实验室）；California Institute of Technology（加利福尼亚理工学院）
In this paper we describe a machine learning based framework for spacecraft swarm trajectory planning. In particular, we focus on coordinating motions of multi-spacecraft in formation flying through passive relative orbit(PRO) transfers. Accounting for spacecraft dynamics while avoiding collisions between the agents makes spacecraft swarm trajectory planning difficult. Centralized approaches can be used to solve this problem, but are computationally demanding and scale poorly with the number of agents in the swarm. As a result, centralized algorithms are ill-suited for real time trajectory pla…
Georgia Institute of Technology（佐治亚理工学院）；Facebook AI Research（Facebook人工智能研究院）；Google Research（谷歌研究院）
PointGoal navigation has seen significant recent interest and progress, spurred on by the Habitat platform and associated challenge. In this paper, we study PointGoal navigation under both a sample budget (75 million frames) and a compute budget (1 GPU for 1 day). We conduct an extensive set of experiments, cumulatively totaling over 50,000 GPU-hours, that let us identify and discuss a number of ostensibly minor but significant design choices -- the advantage estimation procedure (a key component in training), visual encoder architecture, and a seemingly minor hyper-parameter change. Overall, …
Accurate real-time pose estimation of spacecraft or object in space is a key capability necessary for on-orbit spacecraft servicing and assembly tasks. Pose estimation of objects in space is more challenging than for objects on Earth due to space images containing widely varying illumination conditions, high contrast, and poor resolution in addition to power and mass constraints. In this paper, a convolutional neural network is leveraged to uniquely determine the translation and rotation of an object of interest relative to the camera. The main idea of using CNN model is to assist object track…
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）
Visual Teach and Repeat has shown relative navigation is a robust and efficient solution for autonomous vision-based path following in difficult environments. Adding additional absolute sensors such as Global Navigation Satellite Systems (GNSS) has the potential to expand the domain of Visual Teach and Repeat to environments where the ability to visually localize is not guaranteed. Our method of lazy mapping and delaying estimation until a path-tracking error is needed avoids the need to estimate absolute states. As a result, map optimization is not required and paths can be driven immediately…
Estimation of rigid transformation between two point clouds is a computationally challenging problem in vision-based relative navigation. Targeting a real-time navigation solution utilizing point-cloud and image registration algorithms, this paper develops high-performance avionics for power and resource constrained pose estimation framework. A Field-Programmable Gate Array (FPGA) based embedded architecture is developed to accelerate estimation of relative pose between the point-clouds, aided by image features that correspond to the individual point sets. At algorithmic level, the pose estima…
In this work, we have analyzed the problem of relative pose initialization between two satellites: a chaser and a non-cooperating target. The analysis has been targeted to two close-range methods based on a monocular camera system: the Sharma-Ventura-D'Amico (SVD) method and the silhouette matching method. Both methods are based on a priori knowledge of the target geometry, but neither fiducial markers nor a priori range measurements or state information are needed. The tests were carried out using a 2U CubeSat mock-up as target attached to a motorized rotary stage to simulate its relative mot…
Navigation Around Unknown Space Objects Using Visible-Thermal Image Fusion
Draper Scholar, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology（麻省理工学院航空航天系）；Senior Member of Technical Staff, The Charles Stark Draper Laboratory Inc.（查尔斯·斯塔克·德拉珀实验室公司）；Ford Professor of Engineering, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology（麻省理工学院航空航天系）；Professor Post Tenure, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology（麻省理工学院航空航天系）
Hamilton--Jacobi Reachability for Spacecraft Collision Avoidance
Department of Mechanical Engineering, University of California, Berkeley（加州大学伯克利分校机械工程系）；Aerospace Engineering Program, University of California, Berkeley（加州大学伯克利分校航空航天工程项目）；Department of Mechanical Engineering, National University of Singapore（新加坡国立大学机械工程系）
In this paper, we propose a visual-inertial framework able to efficiently estimate the camera poses of a non-rigid trinocular baseline for long-range depth estimation on-board a fast moving aerial platform. The estimation of the time-varying baseline is based on relative inertial measurements, a photometric relative pose optimizer, and a probabilistic wing model fused in an efficient Extended Kalman Filter (EKF) formulation. The estimated depth measurements can be integrated into a geo-referenced global map to render a reconstruction of the environment useful for local replanning algorithms. B…
Woods Hole Oceanographic Institution（伍兹霍尔海洋研究所）；Massachusetts Institute of Technology（麻省理工学院）
This paper presents a scalable acoustic navigation approach for the unified command, control and coordination of multiple autonomous underwater vehicles (AUVs). Existing multi-AUV operations typically achieve coordination manually, by programming individual vehicles on the surface via radio communications, which becomes impractical with large vehicle numbers; or they require bi-directional inter-vehicle acoustic communications to achieve limited coordination when submerged, with limited scalability due to the physical properties of the acoustic channel. Our approach utilizes a single, periodic…
