This paper presents an approach to exploring a multi-objective reinforcement learning problem with Model-Agnostic Meta-Learning. The environment we used consists of a 2D vehicle equipped with a LIDAR sensor. The goal of the environment is to reach some pre-determined target location but also effectively avoid any obstacles it may find along its path. We also compare this approach against a baseline TD3 solution that attempts to solve the same problem.
In recent years, reinforcement learning and its multi-agent analogue have achieved great success in solving various complex control problems. However, multi-agent reinforcement learning remains challenging both in its theoretical analysis and empirical design of algorithms, especially for large swarms of embodied robotic agents where a definitive toolchain remains part of active research. We use emerging state-of-the-art mean-field control techniques in order to convert many-agent swarm control into more classical single-agent control of distributions. This allows profiting from advances in si…
PYROBOCOP is a lightweight Python-based package for control and optimization of robotic systems described by nonlinear Differential Algebraic Equations (DAEs). In particular, the package can handle systems with contacts that are described by complementarity constraints and provides a general framework for specifying obstacle avoidance constraints. The package performs direct transcription of the DAEs into a set of nonlinear equations by performing orthogonal collocation on finite elements. The resulting optimization problem belongs to the class of Mathematical Programs with Complementarity Con…
We propose factor graph optimization for simultaneous planning, control, and trajectory estimation for collision-free navigation of autonomous systems in environments with moving objects. The proposed online probabilistic motion planning and trajectory estimation navigation technique generates optimal collision-free state and control trajectories for autonomous vehicles when the obstacle motion model is both unknown and known. We evaluate the utility of the algorithm to support future autonomous robotic space missions.
Reaching tasks with random targets and obstacles is a challenging task for robotic manipulators. In this study, we propose a novel model-free reinforcement learning approach based on proximal policy optimization (PPO) for training a deep policy to map the task space to the joint space of a 6-DoF manipulator. To facilitate the training process in a large workspace, we develop an efficient representation of environmental inputs and outputs. The calculation of the distance between obstacles and manipulator links is incorporated into the state representation using a geometry-based method. Addition…
Shanghai Jiao Tong University（上海交通大学）；University of Oxford（牛津大学）；Tencent（腾讯）
Collision avoidance is a widely investigated topic in robotic applications. When applying collision avoidance techniques to a mobile robot, how to deal with the spatial structure of the robot still remains a challenge. In this paper, we design a configuration-aware safe control law by solving a Quadratic Programming (QP) with designed Control Barrier Functions (CBFs) constraints, which can safely navigate a mobile robotic arm to a desired region while avoiding collision with environmental obstacles. The advantage of our approach is that it correctly and in an elegant way incorporates the spati…
Predictive Relative-Velocity Steering for Safe Robotic Manipulator Teleoperation in Dynamic Environments
Zhejiang University（浙江大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Tsinghua University（清华大学）；Institute for Embodied Intelligence and Robotics, Tsinghua University（清华大学具身智能与机器人研究所）；TetraBOT
This paper describes the design of a web based multi agent design for a collision avoidance auto navigation biomimetic submarine for submarine hydroelectricity. The paper describes the nature of the map - topology interface for river bodies and the design of interactive agents for the control of the robotic submarine. The agents are migratory on the web and are designed in XML/html interface with both interactive capabilities and visibility on a map. The paper describes mathematically the user interface and the map definition languages used for the multi agent description
Collision avoidance is one of the most important topics in the robotics field. The goal is to move the robots from initial locations to target locations such that they follow shortest non-colliding paths in the shortest time and with the least amount of energy. In this paper, a distributed and real-time algorithm for dense and complex 2D and 3D environments is proposed. This algorithm uses angular calculations to select the optimal direction for the movement of each robot and it has been shown that these separate calculations lead to a form of cooperative behavior among agents. We evaluated th…
Norwegian University of Science and Technology（挪威科技大学）；Oklahoma State University（俄克拉荷马州立大学）
Path Following and Collision Avoidance, be it for unmanned surface vessels or other autonomous vehicles, are two fundamental guidance problems in robotics. For many decades, they have been subject to academic study, leading to a vast number of proposed approaches. However, they have mostly been treated as separate problems, and have typically relied on non-linear first-principles models with parameters that can only be determined experimentally. The rise of Deep Reinforcement Learning (DRL) in recent years suggests an alternative approach: end-to-end learning of the optimal guidance policy fro…
Mixed integer convex and nonlinear programs, MICP and MINLP, are expressive but require long solving times. Recent work that combines data-driven methods on solver heuristics has shown potential to overcome this issue allowing for applications on larger scale practical problems. To solve mixed-integer bilinear programs online with data-driven methods, several formulations exist including mathematical programming with complementary constraints (MPCC), mixed-integer programming (MIP). In this work, we benchmark the performances of those data-driven schemes on a bookshelf organization problem tha…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；The University of Iowa（爱荷华大学）
Biological evidence shows that animals are capable of evading eminent collision without using depth information, relying solely on looming stimuli. In robotics, collision avoidance among uncooperative vehicles requires measurement of relative distance to the obstacle. Small, low-cost mobile robots and UAVs might be unable to carry distance measuring sensors, like LIDARS and depth cameras. We propose a control framework suitable for a unicycle-like vehicle moving in a 2D plane that achieves collision avoidance. The control strategy is inspired by the reaction of invertebrates to approaching obs…
Berkeley AI Research (BAIR), University of California, Berkeley（加州大学伯克利分校伯克利人工智能研究中心）；OpenAI
Reinforcement learning can enable complex, adaptive behavior to be learned automatically for autonomous robotic platforms. However, practical deployment of reinforcement learning methods must contend with the fact that the training process itself can be unsafe for the robot. In this paper, we consider the specific case of a mobile robot learning to navigate an a priori unknown environment while avoiding collisions. In order to learn collision avoidance, the robot must experience collisions at training time. However, high-speed collisions, even at training time, could damage the robot. A succes…
National Institute of Standards and Technology（美国国家标准与技术研究院）；Colorado State University（科罗拉多州立大学）；University of Colorado（科罗拉多大学）；Massachusetts Institute of Technology（麻省理工学院）；Facebook Reality Labs（Facebook现实实验室）
Collision avoidance algorithms are essential for safe and efficient robot operation among pedestrians. This work proposes using deep reinforcement (RL) learning as a framework to model the complex interactions and cooperation with nearby, decision-making agents, such as pedestrians and other robots. Existing RL-based works assume homogeneity of agent properties, use specific motion models over short timescales, or lack a principled method to handle a large, possibly varying number of agents. Therefore, this work develops an algorithm that learns collision avoidance among a variety of heterogen…
A Multiple Artificial Potential Functions Approach for Collision Avoidance in UAV Systems
Brandenburg University of Technology Cottbus-Senftenberg (BTU C-S)（勃兰登堡工业大学科特布斯-森夫滕贝格分校（BTU C-S））；University of Mons（蒙斯大学）；Fraunhofer Research Institution for Energy Infrastructures and Geothermal Systems IEG（弗劳恩霍夫能源基础设施与地热系统研究所IEG）
Luleå University of Technology（吕勒奥理工大学）；WideFind AB（WideFind公司）
Micro Aerial Vehicles (MAVs) navigation in subterranean environments is gaining attention in the field of aerial robotics, however there are still multiple challenges for collision free navigation in such harsh environments. This article proposes a novel baseline solution for collision free navigation with Nonlinear Model Predictive Control (NMPC). In the proposed method, the MAV is considered as a floating object, where the velocities on the $x$, $y$ axes and the position on altitude are the references for the NMPC to navigate along the tunnel, while the NMPC avoids the collision by consideri…
Robotic Shepherding in Cluttered and Unknown Environments using Control Barrier Functions
Center for Artificial Intelligence and Robotics, New York University Abu Dhabi（人工智能与机器人中心，纽约大学阿布扎比分校）；New York University（纽约大学）；New York University, Brooklyn（纽约大学布鲁克林分校）
Berlin Institute of Technology（柏林工业大学）；Chair Industry Grade Networks and Clouds（工业级网络与云研究室）；Faculty of Electrical Engineering, and Computer Science（电气工程与计算机科学学院）；Berlin Institute of Technology, Berlin, Germany（柏林工业大学（德国柏林））；Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany（柏林工业大学电气工程与计算机科学学院工业级网络与云研究室（德国柏林））
Recently, mobile robots have become important tools in various industries, especially in logistics. Deep reinforcement learning emerged as an alternative planning method to replace overly conservative approaches and promises more efficient and flexible navigation. However, deep reinforcement learning approaches are not suitable for long-range navigation due to their proneness to local minima and lack of long term memory, which hinders its widespread integration into industrial applications of mobile robotics. In this paper, we propose a navigation system incorporating deep-reinforcement-learni…
Stanford University（斯坦福大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. A policy trained with expensive data is rendered useless after making even a minor change to the robot hardware. In this paper, we address the challenging problem of adapting a policy, trained to perform a task, to a novel robotic hardware platform given only few demonstrations of robot motion trajectories on the target robot. We formulate it as a few-shot meta-learning problem where the goal is to find a meta-model that captures the common stru…
A central aspect of robotic motion planning is collision avoidance, where a multitude of different approaches are currently in use. Optimization-based motion planning is one method, that often heavily relies on distance computations between robots and obstacles. These computations can easily become a bottleneck, as they do not scale well with the complexity of the robots or the environment. To improve performance, many different methods suggested to use collision primitives, i.e. simple shapes that approximate the more complex rigid bodies, and that are simpler to compute distances to and from…
Task-Space Singularity Avoidance for Control Affine Systems Using Control Barrier Functions
Department of Mechanical Engineering, University of Maryland, College Park（马里兰大学机械工程系，College Park分校）；Division of Magnetic Manipulation and Particle Research, Weinberg Medical Physics, Inc.（Weinberg医学物理公司磁操控与粒子研究部）；Department of Mechanical Engineering, Johns Hopkins University（约翰霍普金斯大学机械工程系）
This paper presents a framework for the safety-critical control of robotic systems, when safety is defined on safe regions in the configuration space. To maintain safety, we synthesize a safe velocity based on control barrier function theory without relying on a -- potentially complicated -- high-fidelity dynamical model of the robot. Then, we track the safe velocity with a tracking controller. This culminates in model-free safety critical control. We prove theoretical safety guarantees for the proposed method. Finally, we demonstrate that this approach is application-agnostic. We execute an o…
In this paper we present the implementation of a Control Barrier Function (CBF) using a quadratic program (QP) formulation that provides obstacle avoidance for a robotic manipulator arm system. CBF is a control technique that has emerged and developed over the past decade and has been extensively explored in the literature on its mathematical foundations, proof of set invariance and potential applications for a variety of safety-critical control systems. In this work we will look at the design of CBF for the robotic manipulator obstacle avoidance, discuss the selection of the CBF parameters an…
Technical University of Munich（慕尼黑工业大学）；Technische Hochschule Ingolstadt（英戈尔施塔特应用科学大学）
Obstacle avoidance is a fundamental and challenging problem for autonomous navigation of mobile robots. In this paper, we consider the problem of obstacle avoidance in simple 3D environments where the robot has to solely rely on a single monocular camera. In particular, we are interested in solving this problem without relying on localization, mapping, or planning techniques. Most of the existing work consider obstacle avoidance as two separate problems, namely obstacle detection, and control. Inspired by the recent advantages of deep reinforcement learning in Atari games and understanding hig…
Robotics Research Center, IIIT Hyderabad（海得拉巴国际信息技术学院机器人研究中心）；Institute of Technology, University of Tartu（塔尔图大学理工学院）
We consider the problem of an agent/robot with non-holonomic kinematics avoiding many dynamic obstacles. State and velocity noise of both the robot and obstacles as well as the robot's control noise are modelled as non-parametric distributions as often the Gaussian assumptions of noise models are violated in real-world scenarios. Under these assumptions, we formulate a robust MPC that samples robotic controls effectively in a manner that aligns the robot to the goal state while avoiding obstacles under the duress of such non-parametric noise. In particular, the MPC incorporates a distribution …
Imitation learning holds the promise to address challenging robotic tasks such as autonomous navigation. It however requires a human supervisor to oversee the training process and send correct control commands to robots without feedback, which is always prone to error and expensive. To minimize human involvement and avoid manual labeling of data in the robotic autonomous navigation with imitation learning, this paper proposes a novel semi-supervised imitation learning solution based on a multi-sensory design. This solution includes a suboptimal sensor policy based on sensor fusion to automatic…
In this paper, we investigate the obstacle avoidance and navigation problem in the robotic control area. For solving such a problem, we propose revised Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization algorithms with an improved reward shaping technique. We compare the performances between the original DDPG and PPO with the revised version of both on simulations with a real mobile robot and demonstrate that the proposed algorithms achieve better results.
Munich School of Robotics and Machine Intelligence（慕尼黑机器人与机器智能学院）；Technical University of Munich（慕尼黑工业大学）
In this paper, we address the problem of vision-based obstacle avoidance for robotic manipulators. This topic poses challenges for both perception and motion generation. While most work in the field aims at improving one of those aspects, we provide a unified framework for approaching this problem. The main goal of this framework is to connect perception and motion by identifying the relationship between the visual input and the corresponding motion representation. To this end, we propose a method for learning reactive obstacle avoidance policies. We evaluate our method on goal-reaching tasks …
Centre for Robotics Research, Department of Informatics, King’s College London（伦敦国王学院信息学系机器人研究中心）；King’s College London（伦敦国王学院）
With the increase in complexity of robotic systems and the rise in non-expert users, it can be assumed that task constraints are not explicitly known. In tasks where avoiding singularity is critical to its success, this paper provides an approach, especially for non-expert users, for the system to learn the constraints contained in a set of demonstrations, such that they can be used to optimise an autonomous controller to avoid singularity, without having to explicitly know the task constraints. The proposed approach avoids singularity, and thereby unpredictable behaviour when carrying out a t…
This paper presents our method for enabling a UAV quadrotor, equipped with a monocular camera, to autonomously avoid collisions with obstacles in unstructured and unknown indoor environments. When compared to obstacle avoidance in ground vehicular robots, UAV navigation brings in additional challenges because the UAV motion is no more constrained to a well-defined indoor ground or street environment. Horizontal structures in indoor and outdoor environments like decorative items, furnishings, ceiling fans, sign-boards, tree branches etc., also become relevant obstacles unlike those for ground v…
