This article proposes a hierarchical learning architecture for safe data-driven control in unknown environments. We consider a constrained nonlinear dynamical system and assume the availability of state-input trajectories solving control tasks in different environments. In addition to task-invariant system state and input constraints, a parameterized environment model generates task-specific state constraints, which are satisfied by the stored trajectories. Our goal is to use these trajectories to find a safe and high-performing policy for a new task in a new, unknown environment. We propose using the stored data to learn generalizable control strategies. At each time step, based on a local forecast of the new task environment, the learned strategy consists of a target region in the state space and input constraints to guide the system evolution to the target region. These target regions are used as terminal sets by a low-level model predictive controller. We show how to i) design the target sets from past data and then ii) incorporate them into a model predictive control scheme with shifting horizon that ensures safety of the closed-loop system when performing the new task. We prove the feasibility of the resulting control policy, and apply the proposed method to robotic path planning, racing, and computer game applications.
Algorithms for motion planning in unknown environments are generally limited in their ability to reason about the structure of the unobserved environment. As such, current methods generally navigate unknown environments by relying on heuristic methods to choose intermediate objectives along frontiers. We present a unified method that combines map prediction and motion planning for safe, time-efficient autonomous navigation of unknown environments by dynamically-constrained robots. We propose a data-driven method for predicting the map of the unobserved environment, using the robot's observatio…
Otto von Guericke University（奥托·冯·格里克大学）；German Aerospace Center (DLR)（德国航空航天中心（DLR））；Technical University of Darmstadt（达姆施塔特工业大学）
Planning and control for autonomous vehicles usually are hierarchical separated. However, increasing performance demands and operating in highly dynamic environments requires an frequent re-evaluation of the planning and tight integration of control and planning to guarantee safety. We propose an integrated hierarchical predictive control and planning approach to tackle this challenge. Planner and controller are based on the repeated solution of moving horizon optimal control problems. The planner can choose different low-layer controller modes for increased flexibility and performance instead…
Reinforcement Learning Compensated Model Predictive Control for Off-road Driving on Unknown Deformable Terrain
Department of Automotive Engineering, Clemson University（汽车工程系，克莱姆森大学）；Ground Vehicle Systems Center, U.S. Army Combat Capabilities Development Command（地面车辆系统中心，美国陆军作战能力发展指挥部）
University of California at Berkeley（加州大学伯克利分校）；California Institute of Technology（加州理工学院）
We present a Model Predictive Control (MPC) strategy for unknown input-affine nonlinear dynamical systems. A non-parametric method is used to estimate the nonlinear dynamics from observed data. The estimated nonlinear dynamics are then linearized over time varying regions of the state space to construct an Affine Time Varying (ATV) model. Error bounds arising from the estimation and linearization procedure are computed by using sampling techniques. The ATV model and the uncertainty sets are used to design a robust Model Predictive Control (MPC) problem which guarantees safety for the unknown s…
Bipedal Robust Walking on Uneven Footholds: Piecewise Slope LIPM with Discrete Model Predictive Control
Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部）；State Key Laboratory of Robotics and System, Harbin Institute of Technology（哈尔滨工业大学机器人技术与系统国家重点实验室）
Adaptive Legged Locomotion via Online Learning for Model Predictive Control
Department of Aerospace Engineering, University of Michigan（密歇根大学航空航天工程系）；Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）
Actor-Critic Cooperative Compensation to Model Predictive Control for Off-Road Autonomous Vehicles Under Unknown Dynamics
Department of Automotive Engineering, Clemson University（汽车工程系，克莱姆森大学）；Ground Vehicle Systems Center, U.S. Army Combat Capabilities Development Command（地面车辆系统中心，美国陆军作战能力发展指挥部）
Recent studies on quadruped robots have focused on either locomotion or mobile manipulation using a robotic arm. Legged robots can manipulate heavier and larger objects using non-prehensile manipulation primitives, such as planar pushing, to drive the object to the desired location. In this paper, we present a novel hierarchical model predictive control (MPC) for contact optimization of the manipulation task. Using two cascading MPCs, we split the loco-manipulation problem into two parts: the first to optimize both contact force and contact location between the robot and the object, and the se…
The ability to autonomously navigate in unknown environments is important for mobile robots. The map is the core component to achieve this. Most map representations rely on drift-free state estimation and provide a global metric map to navigate. However, in large-scale real-world applications, it's hard to prohibit drifts and compose a globally consistent map quickly. In this paper, a novel representation named, HiTMap, is proposed to enhance the existing map representations. The central idea is to adopt a submap-based hierarchical topology rather than a global metric map so that only a local …
HPTune: Hierarchical Proactive Tuning for Collision-Free Model Predictive Control
The University of Hong Kong（香港大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；University of New South Wales（新南威尔士大学）
We propose a learning-based robust predictive control algorithm that compensates for significant uncertainty in the dynamics for a class of discrete-time systems that are nominally linear with an additive nonlinear component. Such systems commonly model the nonlinear effects of an unknown environment on a nominal system. We optimize over a class of nonlinear feedback policies inspired by certainty equivalent "estimate-and-cancel" control laws pioneered in classical adaptive control to achieve significant performance improvements in the presence of uncertainties of large magnitude, a setting in…
California Institute of Technology（加州理工学院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Dexterity Inc.（德克斯特里蒂公司）
This paper proposes a real-time model predictive control (MPC) scheme to execute multiple tasks using robots over a finite-time horizon. In industrial robotic applications, we must carefully consider multiple constraints for avoiding joint position, velocity, and torque limits. In addition, singularity-free and smooth motions require executing tasks continuously and safely. Instead of formulating nonlinear MPC problems, we devise linear MPC problems using kinematic and dynamic models linearized along nominal trajectories produced by hierarchical controllers. These linear MPC problems are solva…
Hierarchical Task Model Predictive Control for Sequential Mobile Manipulation Tasks
Learning Systems and Robotics Lab（学习系统与机器人实验室）；Technical University of Munich（慕尼黑技术大学）；University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；University of Toronto Robotics Institute（多伦多大学机器人研究所）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与机器智能研究所）；Vector Institute for Artificial Intelligence（人工智能向量研究所）
SM2ITH: Safe Mobile Manipulation with Interactive Human Prediction via Task-Hierarchical Bilevel Model Predictive Control
Department of Computer, Control and Management Engineering, of Sapienza University of Rome（意大利萨皮恩扎大学计算机、控制与管理工程系）；University of Toronto Institute for Aerospace Studies (UTIAS) and the Vector Institute for Artificial Intelligence（多伦多大学航空航天研究所（UTIAS）和向量人工智能研究所）；Learning Systems and Robotics lab at the Technical University of Munich and the Munich Institute for Robotics and Machine Intelligence (MIRMI)（慕尼黑技术大学学习系统与机器人实验室及慕尼黑机器人与机器智能研究所（MIRMI））；School of Computing Science, Faculty of Applied Sciences, Simon Fraser University（西蒙·弗雷泽大学应用科学学院计算机科学系）
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）
We present a control method for improved repetitive path following for a ground vehicle that is geared towards long-term operation where the operating conditions can change over time and are initially unknown. We use weighted Bayesian Linear Regression (wBLR) to model the unknown dynamics, and show how this simple model is more accurate in both its estimate of the mean behaviour and model uncertainty than Gaussian Process Regression and generalizes to novel operating conditions with little or no tuning. In addition, wBLR allows us to use fast adaptation and long-term learning in one, unified f…
Hierarchical Learning Framework for Whole-Body Model Predictive Control of a Real Humanoid Robot
ATR Computational Neuroscience Laboratories（ATR计算神经科学实验室）；NTT Communication Science Laboratories, Nippon Telegraph and Telephone Co.（日本电报电话公司NTT通信科学实验室）；Graduate School of Informatics, Kyoto University（京都大学情报学研究科）
Legged robots have shown remarkable advantages in navigating uneven terrain. However, realizing effective locomotion and manipulation tasks on quadruped robots is still challenging. In addition, object and terrain parameters are generally unknown to the robot in these problems. Therefore, this paper proposes a hierarchical adaptive control framework that enables legged robots to perform loco-manipulation tasks without any given assumption on the object's mass, the friction coefficient, or the slope of the terrain. In our approach, we first present an adaptive manipulation control to regulate t…
National University of Singapore（新加坡国立大学）；University of California, Berkeley（加州大学伯克利分校）；Temasek Laboratories, National University of Singapore（新加坡国立大学淡马锡实验室）；University of British Columbia（不列颠哥伦比亚大学）
Distributed optimization is often widely attempted and innovated as an attractive and preferred methodology to solve large-scale problems effectively in a localized and coordinated manner. Thus, it is noteworthy that the methodology of distributed model predictive control (DMPC) has become a promising approach to achieve effective outcomes, e.g., in decision-making tasks for multi-agent systems. However, the typical deployment of such distributed MPC frameworks would lead to the involvement of nonlinear processes with a large number of nonconvex constraints. To address this important problem, …
HDPlanner: Advancing Autonomous Deployments in Unknown Environments through Hierarchical Decision Networks
National University of Singapore（新加坡国立大学）；College of Design and Engineering, National University of Singapore（新加坡国立大学设计与工程学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）
Hierarchical Motion Planning and Offline Robust Model Predictive Control for Autonomous Vehicles
Automation & Control Institute (ACIN), TU Wien（维也纳技术大学自动化与控制研究所）；Austrian Institute of Technology GmbH (AIT)（奥地利技术研究院）；School of Mechanical Engineering, Kyungpook National University（庆北国立大学机械工程学院）；Faculty of Electrical and Electronic Engineering, Phenikaa University（菲尼卡大学电气与电子工程学院）
TRACE: Coverage Path Planning for Unknown Environments Using Hierarchical Coverage Tree
Jinan University（暨南大学）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Jiao Tong University（上海交通大学）；Wuyi University（五邑大学）；South China University of Technology（华南理工大学）
Unified Hierarchical MPC in Task Executing for Modular Manipulators across Diverse Morphologies
Humanoids and Human Centered Mechatronics Research Line, Istituto Italiano Di Tecnologia (IIT)（意大利理工学院人机一体化机电一体化研究组）；Tencent Robotics X Lab（腾讯机器人实验室）；The University of Hong Kong（香港大学）
Velocity-Form Data-Enabled Predictive Control of Soft Robots under Unknown External Payloads
Department of Mechanical Engineering, Michigan State University（机械工程系，密歇根州立大学）；Department of Mechanical Engineering, University of California Riverside（机械工程系，加州大学河滨分校）；Department of Electrical and Computer Engineering, Michigan State University（电气与计算机工程系，密歇根州立大学）；Department of Electrical and Computer Engineering, University of Michigan（电气与计算机工程系，密歇根大学）
The aim of this paper is to propose a new data-driven control scheme for multi-input-multi-output linear time-invariant systems whose system model are completely unknown. Using a non-minimal input-output realization, the proposed method can be applied to the case where the system order is unknown, provided that its upper bound is known. A workaround against measurement noise is proposed and it is shown through simulation study that the proposed method is superior to the conventional methods when dealing with input/output data corrupted by measurement noise.
This paper investigates the data-driven predictive control problems for a class of continuous-time industrial processes with completely unknown dynamics. The proposed approach employs the data-driven technique to get the system matrices online, using input-output measurements. Then, a model-free predictive control approach is designed to implement the receding-horizon optimization and realize the reference tracking. Feasibility of the proposed algorithm and stability of the closed-loop control systems are analyzed, respectively. Finally, a simulation example is provided to demonstrate the effe…
We propose a hierarchical learning architecture for predictive control in unknown environments. We consider a constrained nonlinear dynamical system and assume the availability of state-input trajectories solving control tasks in different environments. A parameterized environment model generates state constraints specific to each task, which are satisfied by the stored trajectories. Our goal is to find a feasible trajectory for a new task in an unknown environment. From stored data, we learn strategies in the form of target sets in a reduced-order state space. These strategies are applied to …
We present differentiable predictive control (DPC) as a deep learning-based alternative to the explicit model predictive control (MPC) for unknown nonlinear systems. In the DPC framework, a neural state-space model is learned from time-series measurements of the system dynamics. The neural control policy is then optimized via stochastic gradient descent approach by differentiating the MPC loss function through the closed-loop system dynamics model. The proposed DPC method learns model-based control policies with state and input constraints, while supporting time-varying references and constrai…
This paper addresses the problem of management and coordination of energy resources in a typical microgrid, including smart buildings as flexible loads, energy storages, and renewables. The overall goal is to provide a comprehensive and innovative framework to maximize the overall benefit, still accounting for possible requests to change the load profile coming from the grid and leaving every single building or user to balance between servicing those requests and satisfying his own comfort levels. The user involvement in the decision-making process is granted by a management and control soluti…
Abstract-In this manuscript, model predictive control for class of discrete fuzzy large-scale systems subjected to bounded time-varying delay and disturbances is studied. The considered method is Razumikhin for time-varying delay large-scale systems, in which it includes a Lyapunov function associated with the original non-augmented state space of system dynamics in comparison with the Krasovskii method. As a rule, the Razumikhin method has a perfect potential to avoid the inherent complexity of the Krasovskii method especially in the presence of large delays and disturbances. The considered l…
In this paper, we use a nonlinear hierarchical model predictive control (MPC) to stabilize the Segway robot. We also use hardware in the loop (HIL) simulation in order to model the delay response of the wheels' motor and verify the control algorithm. In Two-Wheeled Personal Transportation Robots (TWPTR), changing the center of mass location and value, the nonlinearity of the equations, and the dynamics of the system are topics complicating the control problem. A nonlinear MPC predicts the dynamics of the system and solves the control problem efficiently, but requires the exact information of t…
We present a solution for modeling and online identification for heating, ventilation, and air conditioning (HVAC) control in buildings. Our approach comprises: (a) a resistance-capacitance (RC) model based on first order energy balance for deriving the zone temperature dynamics, and (b) a neural network for modeling HVAC dynamics. State estimation and model identification are simultaneously performed using nonlinear moving horizon estimation (MHE) with physical constraints for system states. We leverage the identified model in model predictive control (MPC) for occupant comfort satisfaction a…
In this paper, the model predictive control is designed for an interval type-2 Takagi-Sugeno (T-S) system with unknown time-varying delay in state and input vectors. The time-varying delay is a weird phenomenon that is appeared in almost all systems. It can make many problems and instability while the system is working. In this paper, the time-varying delay is considered in both states and input vectors and is the sensible difference between the proposed method here and previous algorithms, besides, it is unknown but bounded. To solve the problem, the Razumikhin approach is applied to the prop…
