This paper proposes a distributed design method of controllers having a glocal (global/local) information structure for large-scale network systems. Distributed design, independent design of all subcontrollers that constitute a structured controller, facilitates scalable controller synthesis. While existing distributed design methods confine attention to the decentralized or distributed information structures, this study addresses distributed design of glocal-structured controllers. Glocal control exploits the nature that network system's behavior can typically be represented as a superposition of spatially local fluctuations and global interarea oscillations by incorporating a global coordinating subcontroller with local decentralized subcontrollers. The key idea to distributed design of glocal controllers is to represent the original network system as a hierarchical cascaded system composed of reduced-order models representing the global and local dynamics, referred to as hierarchical model decomposition. Distributed design is achieved by independently designing and implementing subcontrollers for the reduced-order models while preserving the cascade structure. This paper provides a condition for existence of the hierarchical model decomposition, a specific representation of the hierarchical system, a clustering method appropriate for the proposed approach, and a robust extension. Numerical examples of a power grid evidence the practical relevance of the proposed method.
A hierarchical Model Predictive Control (MPC) formulation is presented for coupled discrete-time linear systems with state and input constraints. Compared to a centralized approach, a two-level hierarchical controller, with one controller in the upper-level and one controller per subsystem in the lower-level, can significantly reduce the computational cost associated with MPC. Hierarchical coordination is achieved using adjustable tubes, which are optimized by the upper-level controller and bound permissible lower-level controller deviations from the system trajectories determined by the upper…
Frequency control (FC) enables utility-scale grid-connected hybrid power plants (HPPs) to operate in compliance with grid code requirements while to capture value streams from provision of frequency control services (FCSs). In this paper, a novel hierarchical FC approach is proposed to allow HPPs to provide three types of FCSs, namely fast frequency response (FFR), frequency containment response (FCR) and frequency restoration response (FRR). To accommodate state-of-the-art fast FC, controllers for fast FCSs, such as FFR and FCR, are implemented at asset controllers, while controllers for slow…
The paper introduces a Data-driven Hierarchical Control (DHC) structure to improve performance of systems operating under the effect of system and/or environment uncertainty. The proposed hierarchical approach consists of two parts: 1) A data-driven model identification component to learn a linear approximation between reference signals given to an existing lower-level controller and uncertain time-varying plant outputs. 2) A higher-level controller component that utilizes the identified approximation and wraps around the existing controller for the system to handle modeling errors and environ…
This paper presents a novel architecture for model predictive control (MPC) based indoor climate control of multi-zone buildings to provide energy efficiency. Unlike prior works we do not assume the availability of a high-resolution multi-zone building model, which is challenging to obtain. Instead, the architecture uses a low-resolution model of the building which is divided into a small number of "meta-zones" that can be easily identified using existing data-driven modeling techniques. The proposed architecture is hierarchical. At the higher level, an MPC controller uses the low-resolution m…
Single-lane car-following is a fundamental task in autonomous driving. A desirable car-following controller should keep a reasonable range of distances to the preceding vehicle and do so as smoothly as possible. To achieve this, numerous control methods have been proposed: some only rely on local sensing; others also make use of non-local downstream observations. While local methods are capable of attenuating high-frequency velocity oscillation and are economical to compute, non-local methods can dampen a wider spectrum of oscillatory traffic but incur a larger cost in computing. In this artic…
In this paper we propose a hierarchical controller for autonomous racing where the same vehicle model is used in a two level optimization framework for motion planning. The high-level controller computes a trajectory that minimizes the lap time, and the low-level nonlinear model predictive path following controller tracks the computed trajectory online. Following a computed optimal trajectory avoids online planning and enables fast computational times. The efficiency is further enhanced by the coupling of the two levels through a terminal constraint, computed in the high-level controller. Incl…
In this paper we present a hierarchical scheme to detect cyber-attacks in a hierarchical control architecture for large-scale interconnected systems (LSS). We consider the LSS as a network of physically coupled subsystems, equipped with a two-layer controller: on the local level, decentralized controllers guarantee overall stability and reference tracking; on the supervisory level, a centralized coordinator sets references for the local regulators. We present a scheme to detect attacks that occur at the local level, with malicious agents capable of affecting the local control. The detection sc…
We propose a new reinforcement learning based approach to designing hierarchical linear quadratic regulator (LQR) controllers for heterogeneous linear multi-agent systems with unknown state-space models and separated control objectives. The separation arises from grouping the agents into multiple non-overlapping groups, and defining the control goal as two distinct objectives. The first objective aims to minimize a group-wise block-decentralized LQR function that models group-level mission. The second objective, on the other hand, tries to minimize an LQR function between the average states (c…
Field robot systems have recently been applied to a wide range of research fields. Making such systems more automated, advanced, and activated requires cooperation among heterogeneous robots. Classic control theory is inefficient in managing large-scale complex dynamic systems. Therefore, the supervisory control theory based on discrete event system needs to be introduced to overcome this limitation. In this study, we propose a hybrid systems-based hierarchical control architecture through a supervisory control-based high-level controller and a traditional control-based low-level controller. T…
Otto von Guericke University（奥托·冯·格里克大学）；German Aerospace Center (DLR)（德国航空航天中心（DLR））；Technical University of Darmstadt（达姆施塔特工业大学）
Planning and control for autonomous vehicles usually are hierarchical separated. However, increasing performance demands and operating in highly dynamic environments requires an frequent re-evaluation of the planning and tight integration of control and planning to guarantee safety. We propose an integrated hierarchical predictive control and planning approach to tackle this challenge. Planner and controller are based on the repeated solution of moving horizon optimal control problems. The planner can choose different low-layer controller modes for increased flexibility and performance instead…
In this paper, we study the implementation of a model predictive controller (MPC) for the task of object manipulation in a highly uncertain environment (e.g., picking objects from a semi-flexible array of densely packed bins). As a real-time perception-driven feedback controller, MPC is robust to the uncertainties in this environment. However, our experiment shows MPC cannot control a robot to complete a sequence of motions in a heavily occluded environment due to its myopic nature. It will benefit from adding a high-level policy that adaptively adjusts the optimization problem for MPC.
In this paper, we consider networks of static sensors with integrated sensing and communication capabilities. The goal of the sensors is to propagate their collected information to every other agent in the network and possibly a human operator. Such a task requires constant communication among all agents which may result in collisions and congestion in wireless communication. To mitigate this issue, we impose locally non-interfering connectivity constraints that must be respected by every agent. We show that these constraints along with the requirement of propagating information in the network…
This paper solves the problem of regulating the rotor speed tracking error for wind turbines in the full-load region by an effective robust-adaptive control strategy. The developed controller compensates for the uncertainty in the control input effectiveness caused by a pitch actuator fault, unmeasurable wind disturbance, and nonlinearity in the model. Wind turbines have multi-layer structures such that the high-level structure is nonlinearly coupled through an aggregation of the low-level control authorities. Hence, the control design is divided into two stages. First, an $\mathcal{L}_2$ cont…
We present an approach for safe trajectory planning, where a strategic task related to autonomous racing is learned sample-efficient within a simulation environment. A high-level policy, represented as a neural network, outputs a reward specification that is used within the cost function of a parametric nonlinear model predictive controller (NMPC). By including constraints and vehicle kinematics in the NLP, we are able to guarantee safe and feasible trajectories related to the used model. Compared to classical reinforcement learning (RL), our approach restricts the exploration to safe trajecto…
A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shenyang Aircraft Design and Research Institute (SADRI)（沈阳飞机设计研究所）
A Novel Multi-Timescale Stability-Preserving Hierarchical Reinforcement Learning Controller Framework for Adaptive Control in High-Dimensional Dynamical Systems
Faculty of Electrical Engineering K.N. Toosi University of Technology（法拉比科技大学电气工程学院）；Faculty of Mathematical Sciences Shahid Beheshti University（沙希德·贝赫什提大学数学科学学院）；Department of Mathematics and Computer Science University of Central Oklahoma（中央俄克拉荷马大学数学与计算机科学系）
University of California, Berkeley（加州大学伯克利分校）；Tsinghua University（清华大学）
Autonomous vehicles need to handle various traffic conditions and make safe and efficient decisions and maneuvers. However, on the one hand, a single optimization/sampling-based motion planner cannot efficiently generate safe trajectories in real time, particularly when there are many interactive vehicles near by. On the other hand, end-to-end learning methods cannot assure the safety of the outcomes. To address this challenge, we propose a hierarchical behavior planning framework with a set of low-level safe controllers and a high-level reinforcement learning algorithm (H-CtRL) as a coordinat…
Designing the optimal linear quadratic regulator (LQR) for a large-scale multi-agent system (MAS) is time-consuming since it involves solving a large-size matrix Riccati equation. The situation is further exasperated when the design needs to be done in a model-free way using schemes such as reinforcement learning (RL). To reduce this computational complexity, we decompose the large-scale LQR design problem into multiple smaller-size LQR design problems. We consider the objective function to be specified over an undirected graph, and cast the decomposition as a graph clustering problem. The gra…
