In this paper, a MIMO simulated annealing SA based Q learning method is proposed to control a line follower robot. The conventional controller for these types of robots is the proportional P controller. Considering the unknown mechanical characteristics of the robot and uncertainties such as friction and slippery surfaces, system modeling and controller designing can be extremely challenging. The mathematical modeling for the robot is presented in this paper, and a simulator is designed based on this model. The basic Q learning methods are based pure exploitation and the epsilon-greedy methods, which help exploration, can harm the controller performance after learning completion by exploring nonoptimal actions. The simulated annealing based Q learning method tackles this drawback by decreasing the exploration rate when the learning increases. The simulation and experimental results are provided to evaluate the effectiveness of the proposed controller.
Norwegian University of Science and Technology（挪威科技大学）；Oklahoma State University（俄克拉荷马州立大学）
Control theory provides engineers with a multitude of tools to design controllers that manipulate the closed-loop behavior and stability of dynamical systems. These methods rely heavily on insights about the mathematical model governing the physical system. However, in complex systems, such as autonomous underwater vehicles performing the dual objective of path-following and collision avoidance, decision making becomes non-trivial. We propose a solution using state-of-the-art Deep Reinforcement Learning (DRL) techniques, to develop autonomous agents capable of achieving this hybrid objective w…
Communication-Free Collective Navigation for a Swarm of UAVs via LiDAR-Based Deep Reinforcement Learning
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology（机械工程系，釜山国立科学技术研究院）；Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology（机械工程系，韩国科学技术院）
KIOS Research and Innovation Center of Excellence（KIOS卓越研究与创新中心）；University of Cyprus（塞浦路斯大学）
While Unmanned Aerial Vehicles (UAVs) are increasingly deployed in several missions, their inability of reliable and consistent autonomous landing poses a major setback for deploying such systems truly autonomously. In this paper we present an autonomous UAV landing system for landing on a moving platform. In contrast to existing attempts, the proposed system relies only on the camera sensor, and has been designed as lightweight as possible. The proposed system can be deployed on a low power platform as part of the drone payload, whilst being indifferent to any external communication or any ot…
Teams of autonomous unmanned aircraft can be used to monitor wildfires, enabling firefighters to make informed decisions. However, controlling multiple autonomous fixed-wing aircraft to maximize forest fire coverage is a complex problem. The state space is high dimensional, the fire propagates stochastically, the sensor information is imperfect, and the aircraft must coordinate with each other to accomplish their mission. This work presents two deep reinforcement learning approaches for training decentralized controllers that accommodate the high dimensionality and uncertainty inherent in the …
In this paper, a time-varying leader-follower formation control of nonholonomic mobile robots based on a trajectory tracking control strategy is considered. In the time-varying formation, the relative bearing and distance of each follower are variable parameters, and therefore, the followers can carry out various and complex behaviour even without changing the linear and angular velocities of the leader robot. After proposing the kinematic model of the time-varying leader-follower formation, the backstepping control method is exploited to keep the structure of the defined formation. The global…
Alibaba Group（阿里巴巴集团）；The Hong Kong University of Science and Technology（香港科技大学）
Drifting is a complicated task for autonomous vehicle control. Most traditional methods in this area are based on motion equations derived by the understanding of vehicle dynamics, which is difficult to be modeled precisely. We propose a robust drift controller without explicit motion equations, which is based on the latest model-free deep reinforcement learning algorithm soft actor-critic. The drift control problem is formulated as a trajectory following task, where the errorbased state and reward are designed. After being trained on tracks with different levels of difficulty, our controller …
Air traffic control is a real-time safety-critical decision making process in highly dynamic and stochastic environments. In today's aviation practice, a human air traffic controller monitors and directs many aircraft flying through its designated airspace sector. With the fast growing air traffic complexity in traditional (commercial airliners) and low-altitude (drones and eVTOL aircraft) airspace, an autonomous air traffic control system is needed to accommodate high density air traffic and ensure safe separation between aircraft. We propose a deep multi-agent reinforcement learning framewor…
Air traffic control is becoming a more and more complex task due to the increasing number of aircraft. Current air traffic control methods are not suitable for managing this increased traffic. Autonomous air traffic control is deemed a promising alternative. In this paper an air traffic control model is presented that guides an arbitrary number of aircraft across a three-dimensional, unstructured airspace while avoiding conflicts and collisions. This is done utilizing the power of graph based deep learning approaches. These approaches offer significant advantages over current approaches to thi…
In this paper, modification of the classical PID controller and development of open-loop control mechanisms to improve stability and robustness of a differential wheeled robot are discussed. To deploy the algorithm, a test platform has been constructed using low-cost and off-the-shelf components including a microcontroller, reflectance sensor, and motor driver. This paper describes the heuristic approach used in the identification of the system specifications as well as the optimization of the controller. The PID controller is analyzed in detail and the effect of each term is explained in the …
College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学与技术学院）
Developing the flocking behavior for a dynamic squad of fixed-wing UAVs is still a challenge due to kinematic complexity and environmental uncertainty. In this paper, we deal with the decentralized flocking and collision avoidance problem through deep reinforcement learning (DRL). Specifically, we formulate a decentralized DRL-based decision making framework from the perspective of every follower, where a collision avoidance mechanism is integrated into the flocking controller. Then, we propose a novel reinforcement learning algorithm PS-CACER for training a shared control policy for all the f…
Oden Institute for Computational Engineering and Sciences, University of Texas, Austin（得克萨斯大学奥斯汀分校奥登计算工程与科学研究所）；University of Florida（佛罗里达大学）
This paper develops a controller synthesis approach for a multi-agent system (MAS) with intermittent communication. We adopt a leader-follower scheme, where a mobile leader with absolute position sensors switches among a set of followers without absolute position sensors to provide each follower with intermittent state information.We model the MAS as a switched system. The followers are to asymptotically reach a predetermined consensus state. To guarantee the stability of the switched system and the consensus of the followers, we derive maximum and minimal dwell-time conditions to constrain th…
Missouri University of Science and Technology（密苏里科技大学）
Mobile and slow moving operations, such as striping, sweeping, bridge flushing and pothole patching, are critical for efficient and safe operation of the highway transportation system. A successfully implemented leader follower autonomous truck mounted attenuators system will eliminate all injuries to DOT employees in follow truck provided appropriate Statutory authority. The leader follower system design imposes more requirements to the lead truck drivers in order to ensure a safe and smooth system operation. The driver is now required to make driving decisions not only from the lead truck's …
A leader-follower system is developed for cooperative transportation. To the best of our knowledge, this is the first work that inter-UAV communication is not required and the reference trajectory of the payload can be modified in real time, so that it can be applied to a dynamically changing environment. To track the modified reference trajectory in real time under the communication-free condition, the leader-follower system is considered as a nonholonomic system in which a controller is developed for the leader to achieve asymptotic tracking of the payload. To eliminate the need to install f…
University of Florida（佛罗里达大学）；Amazon Robotics（亚马逊机器人公司）；Tongdun Technology（同盾科技）
We propose a deep reinforcement learning (DRL) methodology for the tracking, obstacle avoidance, and formation control of nonholonomic robots. By separating vision-based control into a perception module and a controller module, we can train a DRL agent without sophisticated physics or 3D modeling. In addition, the modular framework averts daunting retrains of an image-to-action end-to-end neural network, and provides flexibility in transferring the controller to different robots. First, we train a convolutional neural network (CNN) to accurately localize in an indoor setting with dynamic foreg…
Distributed 3D Leader-Follower Formation Control with Field-of-View Safety via Control Barrier Functions
Department of Electrical and Computer Engineering, University of Houston（休斯敦大学电气与计算机工程系）；Department of Engineering Technology, University of Houston（休斯敦大学工程技术系）
The Autonomous Truck-Mounted Attenuator (ATMA) system is a lead-follower vehicle system based on autonomous driving and connected vehicle technologies. The lead truck performs maintenance tasks on the road, and the unmanned follower truck alerts passing vehicles about the moving work zone and protects workers and the equipment. While the ATMA has been under testing by transportation maintenance and operations agencies recently, a simulator-based testing capability is a supplement, especially if human subjects are involved. This paper aims to discover how passing drivers perceive, understand, a…
In this tutorial, we detailed simple controllers for autonomous parking and path following for self-driving cars providing practical methods for curvature computation.
University of Michigan（密歇根大学）；Ford Motor Company（福特汽车公司）
Merging is, in general, a challenging task for both human drivers and autonomous vehicles, especially in dense traffic, because the merging vehicle typically needs to interact with other vehicles to identify or create a gap and safely merge into. In this paper, we consider the problem of autonomous vehicle control for forced merge scenarios. We propose a novel game-theoretic controller, called the Leader-Follower Game Controller (LFGC), in which the interactions between the autonomous ego vehicle and other vehicles with a priori uncertain driving intentions is modeled as a partially observable…
A quadcopter is an under-actuated system with only four control inputs for six degrees of freedom, and yet the human control of a quadcopter is simple enough to be learned with some practice. In this work, we consider the problem of human control of a multiple quadcopters system to transport a cable-suspended payload. The coupled dynamics of the system, due to the inherent physical constraints, is used to develop a leader-follower architecture where the leader quadcopter is controlled directly by a human operator and the followers are controlled with the proposed Payload Attitude Controller an…
Southern University of Science and Technology（南方科技大学）
Coordinated motion control in swarm robotics aims to ensure the coherence of members in space, i.e., the robots in a swarm perform coordinated movements to maintain spatial structures. This problem can be modeled as a tracking control problem, in which individuals in the swarm follow a target position with the consideration of specific relative distance or orientations. To keep the communication cost low, the PID controller can be utilized to achieve the leader-follower tracking control task without the information of leader velocities. However, the controller's parameters need to be optimized…
Tampere University（坦佩雷大学）；Aalto University（阿尔托大学）；Technische Universität Darmstadt（达姆施塔特工业大学）；Cargotec Oyj（卡哥特科公司）
We have recently proposed two pile loading controllers that learn from human demonstrations: a neural network (NNet) [1] and a random forest (RF) controller [2]. In the field experiments the RF controller obtained clearly better success rates. In this work, the previous findings are drastically revised by experimenting summer time trained controllers in winter conditions. The winter experiments revealed a need for additional sensors, more training data, and a controller that can take advantage of these. Therefore, we propose a revised neural controller (NNetV2) which has a more expressive stru…
Multi-Agent gatekeeper: Safe Flight Planning and Formation Control for Urban Air Mobility
Department of Electrical & Computer Engineering（电气与计算机工程系）；Department of Robotics（机器人学系）；Department of Robotics and Department of Aerospace Engineering（机器人学系和航空航天工程系）
IIT Kharagpur（印度理工学院卡拉格普尔分校）；Carnegie Mellon University（卡内基梅隆大学）；Cleveland State University（克利夫兰州立大学）
Delays endanger safety of autonomous systems operating in a rapidly changing environment, such as nondeterministic surrounding traffic participants in autonomous driving and high-speed racing. Unfortunately, delays are typically not considered during the conventional controller design or learning-enabled controller training phases prior to deployment in the physical world. In this paper, the computation delay from nonlinear optimization for motion planning and control, as well as other unavoidable delays caused by actuators, are addressed systematically and unifiedly. To deal with all these de…
University of North Carolina Chapel Hill（北卡罗来纳大学教堂山分校）
In this paper, we perform safety and performance analysis of an autonomous vehicle that implements reactive planner and controller for navigating a race lap. Unlike traditional planning algorithms that have access to a map of the environment, reactive planner generates the plan purely based on the current input from sensors. Our reactive planner selects a waypoint on the local Voronoi diagram and we use a pure-pursuit controller to navigate towards the waypoint. Our safety and performance analysis has two parts. The first part demonstrates that the reactive planner computes a plan that is loca…
Vision-Based Leader-Follower Formation Control for Cooperative UAVs in GPS-Degraded Environments
Autonomous Robotics Systems Limited（自主机器人系统有限公司）；University of Hyderabad（海得拉巴大学）；Ideabytes Software India Private Limited（Ideabytes软件印度私人有限公司）；Georgia Institute of Technology（佐治亚理工学院）；Tata Consultancy Services（塔塔咨询服务公司）；University of Malaya（马来亚大学）
University of Leuven (KU Leuven)（鲁汶大学）；Bogazici University（博阿齐奇大学）
Provision of some autonomous functions to an agricultural vehicle would lighten the job of the operator but in doing so, the accuracy should not be lost to still obtain an optimal yield. Autonomous navigation of an agricultural vehicle involves the control of different dynamic subsystems, such as the yaw angle dynamics and the longitudinal speed dynamics. In this study, a proportional-integral-derivative controller is used to control the longitudinal velocity of the tractor. For the control of the yaw angle dynamics, a proportional-derivative controller works in parallel with a type-2 fuzzy ne…
Nowadays an accurate modeling of the system to be controlled is essential for reliable autopilot. This paper presents a non-linear model of the autonomous underwater vehicle 'CWolf'. Matrices and the corresponding coefficients generate a parameterized representation for added mass, Coriolis and centripetal forces, damping, gravity and buoyancy, using the equations of motion, for all six degrees of freedom. The determination of actuator behaviour by surge tests allows the conversion of propeller revolutions to the respective forces and moments. Based on geometric approximations, the coefficient…
This paper presents an adaptive lookahead pure-pursuit lateral controller for optimizing racing metrics such as lap time, average lap speed, and deviation from a reference trajectory in an autonomous racing scenario. We propose a greedy algorithm to compute and assign optimal lookahead distances for the pure-pursuit controller for each waypoint on a reference trajectory for improving the race metrics. We use a ROS based autonomous racing simulator to evaluate the adaptive pure-pursuit algorithm and compare our method with several other pure-pursuit based lateral controllers. We also demonstrat…
Interactions between road users are both highly non-linear and profoundly complex, and there is no reason to expect that interactions between autonomous vehicles will be any different. Given the recent rapid development of autonomous vehicle technologies, we need to understand how these interactions are likely to present themselves, and what their implications might be. This paper looks into the impact of autonomous vehicles with differing controllers, focusing specifically on the effects of changing the mean and heterogeneity of controller parameters on three key performance metrics: throughp…
Norwegian University of Science and Technology（挪威科技大学）；SINTEF Digital（挪威科技工业研究院数字部门）；Oklahoma State University（俄克拉荷马州立大学）
Control theory provides engineers with a multitude of tools to design controllers that manipulate the closed-loop behavior and stability of dynamical systems. These methods rely heavily on insights about the mathematical model governing the physical system. However, if a system is highly complex, it might be infeasible to produce a reliable mathematical model of the system. Without a model most of the theoretical tools to develop control laws break down. In these settings, machine learning controllers become attractive: Controllers that can learn and adapt to complex systems, developing contro…
Massachusetts Institute of Technology（麻省理工学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；KU Leuven（鲁汶大学）；Flanders Make（法兰德斯制造研究院）
Accurate control of autonomous marine robots still poses challenges due to the complex dynamics of the environment. In this paper, we propose a Deep Reinforcement Learning (DRL) approach to train a controller for autonomous surface vessel (ASV) trajectory tracking and compare its performance with an advanced nonlinear model predictive controller (NMPC) in real environments. Taking into account environmental disturbances (e.g., wind, waves, and currents), noisy measurements, and non-ideal actuators presented in the physical ASV, several effective reward functions for DRL tracking control polici…
Massachusetts Institute of Technology (MIT)（麻省理工学院）
Modern robotic manipulation systems fall short of human manipulation skills partly because they rely on closing feedback loops exclusively around vision data, which reduces system bandwidth and speed. By developing autonomous grasping reflexes that rely on high-bandwidth force, contact, and proximity data, the overall system speed and robustness can be increased while reducing reliance on vision data. We are developing a new system built around a low-inertia, high-speed arm with nimble fingers that combines a high-level trajectory planner operating at less than 1 Hz with low-level autonomous r…
The main objective of this work is demonstrated through two main aspects. The first is the design of an adaptive neuro-fuzzy inference system (ANFIS) controller to develop the attitude and altitude of a quadcopter. The second is to establish the linearized mathematical model of the quadcopter in a simple and clear way. To show the effectiveness of the ANFIS approach, the performance of a well-trained ANFIS controller is compared to a classical proportional-derivative (PD) controller and a properly tuned fuzzy logic controller.
Australian Centre for Field Robotics, The University of Sydney（悉尼大学澳大利亚野外机器人中心）
Social acceptance is a major hurdle for autonomous vehicle technology, central to which is ensuring both passengers and nearby pedestrians feel safe. This idea of `feeling safe' and perceived safety is highly subjective and rooted in human intuition. As such, traditional analytical approaches to autonomous navigation often fail to cater for the social expectations of individuals. Therefore, this paper proposes an approach to capture the complexity of social expectations and integrate this complexity into a 3-layered Contextual Speed Controller. The layers were; the legal road speed limit, the …
Indian Institute of Science（印度科学学院）；WIRIN Lab（WIRIN实验室）
This paper presents a coupled, neural network-aided longitudinal cruise and lateral path-tracking controller for an autonomous vehicle with model uncertainties and experiencing unknown external disturbances. Using a feedback error learning mechanism, an inverse vehicle dynamics learning scheme utilizing an adaptive Radial Basis Function (RBF) neural network, referred to as the Extended Minimal Resource Allocating Network (EMRAN) is employed. EMRAN uses an extended Kalman filter for online learning and weight updates, and also incorporates a growing/pruning strategy for maintaining a compact ne…
This paper describes Ariel Team's autonomous racing controller for the Indy Autonomous Challenge (IAC) simulation race. IAC is the first multi-vehicle autonomous head-to-head competition, reaching speeds of 300 km/h along an oval track, modeled after the Indianapolis Motor Speedway (IMS). Our racing controller attempts to maximize progress along the track while avoiding collisions with opponent vehicles and obeying the race rules. To this end, the racing controller first computes a race line offline. Then, it repeatedly computes online a small set of dynamically feasible maneuver candidates, e…
Actor-Critic Cooperative Compensation to Model Predictive Control for Off-Road Autonomous Vehicles Under Unknown Dynamics
Department of Automotive Engineering, Clemson University（汽车工程系，克莱姆森大学）；Ground Vehicle Systems Center, U.S. Army Combat Capabilities Development Command（地面车辆系统中心，美国陆军作战能力发展指挥部）
Verified controller synthesis uses world models that comprise all potential behaviours of humans, robots, further equipment, and the controller to be synthesised. A world model enables quantitative risk assessment, for example, by stochastic model checking. Such a model describes a range of controller behaviours some of which -- when implemented correctly -- guarantee that the overall risk in the actual world is acceptable, provided that the stochastic assumptions have been made to the safe side. Synthesis then selects an acceptable-risk controller behaviour. However, because of crossing abstr…
Shanghai University（上海大学）；Michigan State University（密歇根州立大学）
Deep reinforcement learning (DRL) has a great potential for solving complex decision-making problems in autonomous driving, especially in mixed-traffic scenarios where autonomous vehicles and human-driven vehicles (HDVs) drive together. Safety is a key during both the learning and deploying reinforcement learning (RL) algorithms process. In this paper, we formulate the on-ramp merging as a Markov Decision Process (MDP) problem and solve it with an off-policy RL algorithm, i.e., Soft Actor-Critic for Discrete Action Settings (SAC-Discrete). In addition, a motion predictive safety controller inc…
This paper addresses the problem of cooperative transport of a point mass hoisted by two aerial robots. Treating the robots as a leader and a follower, the follower stabilizes the system with respect to the leader using only feedback from its Inertial Measurement Units (IMU). This is accomplished by neglecting the acceleration of the leader, analyzing the system through the generalized coordinates or the cables' angles, and employing an observation model based on the IMU measurements. A lightweight estimator based on an Extended Kalman Filter (EKF) and a controller are derived to stabilize the…
Northwestern Polytechnical University（西北工业大学）；City University of London（伦敦城市大学）
Autonomous navigation in unknown complex environment is still a hard problem, especially for small Unmanned Aerial Vehicles (UAVs) with limited computation resources. In this paper, a neural network-based reactive controller is proposed for a quadrotor to fly autonomously in unknown outdoor environment. The navigation controller makes use of only current sensor data to generate the control signal without any optimization or configuration space searching, which reduces both memory and computation requirement. The navigation problem is modelled as a Markov Decision Process (MDP) and solved using…
Chaitanya Bharathi Institute of Technology（查塔尼亚·巴拉蒂理工学院）
There are few industries which use manually controlled robots for carrying material and this cannot be used all the time in all the places. So, it is very tranquil to have robots which can follow a specific human by following the unique coloured object held by that person. So, we propose a robotic system which uses robot vision and deep learning to get the required linear and angular velocities which are ν and ω, respectively. Which in turn makes the robot to avoid obstacles when following the unique coloured object held by the human. The novel methodology that we are proposing is accurate in …
Avoiding unseen or partially occluded vulnerable road users (VRUs) is a major challenge for fully autonomous driving in urban scenes. However, occlusion-aware risk assessment systems have not been widely studied. Here, we propose a pedestrian emergence estimation and occlusion-aware risk assessment system for urban autonomous driving. First, the proposed system utilizes available contextual information, such as visible cars and pedestrians, to estimate pedestrian emergence probabilities in occluded regions. These probabilities are then used in a risk assessment framework, and incorporated into…
Lab-STICC（Lab-STICC实验室）；ENSTA-Bretagne（布列塔尼国立高等先进技术学院）；LIX（LIX实验室）；Ecole Polytechnique（巴黎综合理工学院）；CNRS（法国国家科学研究中心）；Institut Polytechnique de Paris（巴黎理工学院）
We study learning based controllers as a replacement for model predictive controllers (MPC) for the control of autonomous vehicles. We concentrate for the experiments on the simple yet representative bicycle model. We compare training by supervised learning and by reinforcement learning. We also discuss the neural net architectures so as to obtain small nets with the best performances. This work aims at producing controllers that can both be embedded on real-time platforms and amenable to verification by formal methods techniques.
