Docking control of an autonomous underwater vehicle (AUV) is a task that is integral to achieving persistent long term autonomy. This work explores the application of state-of-the-art model-free deep reinforcement learning (DRL) approaches to the task of AUV docking in the continuous domain. We provide a detailed formulation of the reward function, utilized to successfully dock the AUV onto a fixed docking platform. A major contribution that distinguishes our work from the previous approaches is the usage of a physics simulator to define and simulate the underwater environment as well as the DeepLeng AUV. We propose a new reward function formulation for the docking task, incorporating several components, that outperforms previous reward formulations. We evaluate proximal policy optimization (PPO), twin delayed deep deterministic policy gradients (TD3) and soft actor-critic (SAC) in combination with our reward function. Our evaluation yielded results that conclusively show the TD3 agent to be most efficient and consistent in terms of docking the AUV, over multiple evaluation runs it achieved a 100% success rate and episode return of 10667.1 +- 688.8. We also show how our reward function formulation improves over the state of the art.
MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics
The State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University（浙江大学流体动力与机电系统国家重点实验室）；School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）；School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Department of Mechanical and Aerospace Engineering, the Hong Kong University of Science and Technology（香港科技大学机械与航空航天工程系）
The design of autonomous underwater vehicles (AUVs) and their docking stations has been a popular research topic for several decades. Although many AUV and dock designs have been proposed, materialized, and commercialized, most of these existing designs prioritize the functionality of the AUV over the dock, or vise versa; there has been limited formal research in analytical optimization for AUV docking systems. In this paper, a multidisciplinary optimization framework is presented with the aim to fill this theoretical gap. We propose a co-design optimization method that optimizes multiple desi…
AquaticVision: Benchmarking Visual SLAM in Underwater Environment with Events and Frames
School of Data Science, The Chinese University of Hong Kong, Shenzhen, P. R. China（数据科学学院，香港中文大学（深圳））；School of Life Sciences, The Chinese University of Hong Kong, Hongkong（生命科学学院，香港中文大学）
STERN: Simultaneous Trajectory Estimation and Relative Navigation for Autonomous Underwater Proximity Operations
Centre for Naval Architecture, KTH Royal Institute of Technology（KTH皇家理工学院海军建筑中心）；Division of Robotics, Perception, and Learning, KTH Royal Institute of Technology（KTH皇家理工学院机器人、感知与学习系）
Relative Navigation and Dynamic Target Tracking for Autonomous Underwater Proximity Operations
AI, Robotics and Cybersecurity Center, Örebro University（人工智能、机器人与网络安全中心，奥雷布罗大学）；Centre for Naval Architecture, KTH Royal Institute of Technology（海军工程中心，皇家理工学院）；department of Aeronautics and Astronautics, Massachusetts Institute of Technology（航空与航天系，麻省理工学院）；Division of Robotics, Perception and Learning, KTH Royal Institute of Technology（机器人、感知与学习系，皇家理工学院）
SeaVis: Modeling and Control of a Remotely Operated Towed Vehicle for Seabed Visualization and Mapping
Artificial Intelligence in Robotics Laboratory (AiR Lab)（人工智能机器人实验室（AiR实验室））；Department of Electrical and Computer Engineering（电气与计算机工程系）；EIVA a/s（EIVA公司）；Automatic Control Group (RAT)（自动控制组（RAT））；Department of Electrical Engineering and Information Technology（电气工程与信息科技系）
