In this paper, we present an exploration and assessment of employing a centralized deep Q-network (DQN) controller as a substitute for the prevalent use of PID controllers in the context of 6DOF swimming robots. Our primary focus centers on illustrating this transition with the specific case of underwater object tracking. DQN offers advantages such as data efficiency and off-policy learning, while remaining simpler to implement than other reinforcement learning methods. Given the absence of a dynamic model for our robot, we propose an RL agent to control this multi-input-multi-output (MIMO) system, where a centralized controller may offer more robust control than distinct PIDs. Our approach involves initially using classical controllers for safe exploration, then gradually shifting to DQN to take full control of the robot.
We divide the underwater tracking task into vision and control modules. We use established methods for vision-based tracking and introduce a centralized DQN controller. By transmitting bounding box data from the vision module to the control module, we enable adaptation to various objects and effortless vision system replacement. Furthermore, dealing with low-dimensional data facilitates cost-effective online learning for the controller. Our experiments, conducted within a Unity-based simulator, validate the effectiveness of a centralized RL agent over separated PID controllers, showcasing the applicability of our framework for training the underwater RL agent and improved performance compared to traditional control methods. The code for both real and simulation implementations is at https://github.com/FARAZLOTFI/underwater-object-tracking.
How do trout regulate patterns of muscle contraction to optimize propulsive efficiency during steady swimming
The College of River and Ocean Engineering, Chongqing Jiaotong University,Chongqing, 400016, China（重庆交通大学河流海洋工程学院）；State Key Laboratory of Hydraulics and Mountain River Engineering, Sichuan University, Chengdu, 610065, China（四川大学水利与山区河流工程重点实验室）；Chongqing Deepflow Intelligent Computing Technology Co., Ltd, Chongqing, 401122, China（重庆深流智能计算技术有限公司）
Swimming Under Constraints: A Safe Reinforcement Learning Framework for Quadrupedal Bio-Inspired Propulsion
FSI lab, Westlake University（西交大实验室，西湖大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Zhejiang University（浙江大学）；Zhongguancun Academy（中关村学院）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）
Faced with strong demand for robots working in underwater pipeline environments, a novel underwater multi-model locomotion robot is designed and studied in this research. By mimicking the earthworm's metameric body, the robot is segmented in the structure; by synthesizing the earthworm-like peristaltic locomotion mechanism and the propeller-driven swimming mechanism, the robot possesses unique multi-mode locomotion capability. In detail, the in-pipe earthworm-like peristaltic crawling is achieved based on servomotor-driven cords and pre-bent spring-steel belts that work antagonistically, and t…
Learning Control as Enabling Layer for Embodied Intelligence Research explored with Soft Robotic Swimming in diverse Flow Speeds
Engineering Sciences Department, Swiss Federal Laboratories for Materials Science and Technology（瑞士联邦材料科学与技术实验室工程科学系）；Institute for NeuroInformatics, University of Zurich（苏黎世大学神经信息研究所）；Department of Electronic Engineering, University of Rome Tor Vergata（罗马大学Tor Vergata电子工程系）
Robotic Classification of Divers' Swimming States using Visual Pose Keypoints as IMUs
Department of Computer Science & Engineering（计算机科学与工程系）；Minnesota Robotics Institute (MnRI)（明尼苏达州机器人研究所）；University of Minnesota–Twin Cities（明尼苏达大学-双城分校）；Department of Computer Science（计算机科学系）；Carnegie Mellon University（卡内基梅隆大学）
Brown University（布朗大学）；Universidad Nacional Autónoma de México（墨西哥国立自治大学）
Marine exploration is essential to understanding ocean processes and organisms. While the use of current unmanned underwater vehicles has enabled many discoveries, there are still plenty of limitations toward exploring complex environments. Bio-inspired robots are a promising solution for highly maneuverable underwater swimming at moderate speeds. Krill, especially, are efficient swimmers in the intermediate Reynolds number regime and can inform engineering solutions for ocean exploration. In this paper, we present the design, manufacture, and validation of a new krill-inspired, metachronal, d…
Nowadays an accurate modeling of the system to be controlled is essential for reliable autopilot. This paper presents a non-linear model of the autonomous underwater vehicle 'CWolf'. Matrices and the corresponding coefficients generate a parameterized representation for added mass, Coriolis and centripetal forces, damping, gravity and buoyancy, using the equations of motion, for all six degrees of freedom. The determination of actuator behaviour by surge tests allows the conversion of propeller revolutions to the respective forces and moments. Based on geometric approximations, the coefficient…
This paper presents an approach for autonomous underwater robots to visually detect and identify divers. The proposed approach enables an autonomous underwater robot to detect multiple divers in a visual scene and distinguish between them. Such methods are useful for robots to identify a human leader, for example, in multi-human/robot teams where only designated individuals are allowed to command or lean a team of robots. Initial diver identification is performed using the Faster R-CNN algorithm with a region proposal network which produces bounding boxes around the divers' locations. Subseque…
University of South Carolina（南卡罗来纳大学）；McGill University（麦吉尔大学）；Dartmouth College（达特茅斯学院）
In this paper, we propose a real-time deep learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to the profound difficulty of collecting ground truth images with accurate 6D poses underwater, this work utilizes rendered images from the Unreal Game Engine simulation for training. An image-to-image tra…
Bionic underwater robots have demonstrated their superiority in many applications. Yet, training their intelligence for a variety of tasks that mimic the behavior of underwater creatures poses a number of challenges in practice, mainly due to lack of a large amount of available training data as well as the high cost in real physical environment. Alternatively, simulation has been considered as a viable and important tool for acquiring datasets in different environments, but it mostly targeted rigid and soft body systems. There is currently dearth of work for more complex fluid systems interact…
In this paper, we present a number of robust methodologies for an underwater robot to visually detect, follow, and interact with a diver for collaborative task execution. We design and develop two autonomous diver-following algorithms, the first of which utilizes both spatial- and frequency-domain features pertaining to human swimming patterns in order to visually track a diver. The second algorithm uses a convolutional neural network-based model for robust tracking-by-detection. In addition, we propose a hand gesture-based human-robot communication framework that is syntactically simpler and …
EPFL（洛桑联邦理工学院）；Scuola Superiore Sant’Anna（圣安娜高等研究院）；Università di Pisa（比萨大学）；Delft University of Technology（代尔夫特理工大学）；German Aerospace Center (DLR)（德国航空航天中心）
The success of soft robots in displaying emergent behaviors is tightly linked to the compliant interaction with the environment. However, to exploit such phenomena, proprioceptive sensing methods which do not hinder their softness are needed. In this work we propose a new sensing approach for soft underwater slender structures based on embedded pressure sensors and use a learning-based pipeline to link the sensor readings to the shape of the soft structure. Using two different modeling techniques, we compare the pose reconstruction accuracy and identify the optimal approach. Using the proprioc…
Jacobs University Bremen gGmbH（不来梅雅各布大学）；National Research Council of Italy (CNR)（意大利国家研究委员会）；University of Zagreb（萨格勒布大学）
In this article we present a novel underwater dataset collected from several field trials within the EU FP7 project "Cognitive autonomous diving buddy (CADDY)", where an Autonomous Underwater Vehicle (AUV) was used to interact with divers and monitor their activities. To our knowledge, this is one of the first efforts to collect a large dataset in underwater environments targeting object classification, segmentation and human pose estimation tasks. The first part of the dataset contains stereo camera recordings (~10K) of divers performing hand gestures to communicate and interact with an AUV i…
Southampton Marine and Maritime Institute, University of Southampton（南安普顿大学南安普顿海洋与海事研究所）；Alan Turing Institute（艾伦·图灵研究所）；Institute of Industrial Science, The University of Tokyo（东京大学生产技术研究所）
The optimal stiffness for soft swimming robots depends on swimming speed, which means no single stiffness can maximise efficiency in all swimming conditions. Tunable stiffness would produce an increased range of high-efficiency swimming speeds for robots with flexible propulsors and enable soft control surfaces for steering underwater vehicles. We propose and demonstrate a method for tunable soft robotic stiffness using inflatable rubber tubes to stiffen a silicone foil through pressure and second moment of area change. We achieved double the effective stiffness of the system for an input pres…
Norwegian University of Science and Technology（挪威科技大学）；SINTEF Digital（挪威科技工业研究院数字部门）；Oklahoma State University（俄克拉荷马州立大学）
Control theory provides engineers with a multitude of tools to design controllers that manipulate the closed-loop behavior and stability of dynamical systems. These methods rely heavily on insights about the mathematical model governing the physical system. However, if a system is highly complex, it might be infeasible to produce a reliable mathematical model of the system. Without a model most of the theoretical tools to develop control laws break down. In these settings, machine learning controllers become attractive: Controllers that can learn and adapt to complex systems, developing contro…
Learn to Swim: Data-Driven LSTM Hydrodynamic Model for Quadruped Robot Gait Optimization
Zhejiang University（浙江大学）；FSI lab, Westlake University（西湖大学FSI实验室）；Xinyang Normal University（新阳师范学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Unified Structural-Hydrodynamic Modeling of Underwater Underactuated Mechanisms and Soft Robots
Department of Mechanical Engineering and Advanced Robotics Centre, National University of Singapore（新加坡国立大学机械工程系与先进机器人中心）；Singapore-MIT Alliance for Research and Technology (SMART) Centre（新加坡-麻省理工学院研究与技术联盟中心）
Never too Prim to Swim: An LLM-Enhanced RL-based Adaptive S-Surface Controller for AUVs under Extreme Sea Conditions
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China（清华大学深圳国际研究生院，清华大学，深圳，518055，中国）；Department of Data Science, New Jersey Institute of Technology（数据科学系，新泽西理工学院）
University of Maryland at College Park（马里兰大学帕克分校）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
We present an efficient algorithm for motion planning and control of a robot system with a high number of degrees-of-freedom. These include high-DOF soft robots or an articulated robot interacting with a deformable environment. Our approach takes into account dynamics constraints and present a novel technique to accelerate the forward dynamic computation using a data-driven method. We precompute the forward dynamic function of the robot system on a hierarchical adaptive grid. Furthermore, we exploit the properties of underactuated robot systems and perform these computations for a few DOFs. We…
SeaVis: Modeling and Control of a Remotely Operated Towed Vehicle for Seabed Visualization and Mapping
Artificial Intelligence in Robotics Laboratory (AiR Lab)（人工智能机器人实验室（AiR实验室））；Department of Electrical and Computer Engineering（电气与计算机工程系）；EIVA a/s（EIVA公司）；Automatic Control Group (RAT)（自动控制组（RAT））；Department of Electrical Engineering and Information Technology（电气工程与信息科技系）
