安全性一直是基于学习的方法在实际应用中部署的关键问题。为解决这一问题，控制屏障函数（control barrier function, CBF）及其变体在安全关键控制领域受到了广泛关注。然而，由于CBF的短视单步特性，以及缺乏设计$\boldsymbol{\text{class-}\boldsymbol{\text{K}}}$函数的原则性方法，当前的CBF仍存在最优性、稳定性和可行性方面的根本局限。本文提出了一种新颖的统一方法——自适应多步控制屏障函数（Adaptive Multi-step Control Barrier Function, AM-CBF）来解决这些局限，我们用神经网络对$\text{class-}\text{K}$函数进行参数化，并将其与强化学习策略联合训练。此外，为缓解短视特性，我们提出了一种新颖的「多步训练、单步执行」范式，让CBF具备远见性，同时执行阶段仍只需求解单步凸二次规划。我们在多种场景下的一阶和二阶系统上评估了该方法，结果表明我们的方法在定性和定量层面均优于传统CBF。
英文摘要：
Safety has been a critical issue for the deployment of learning-based approaches in real-world applications. To address this issue, control barrier function (CBF) and its variants have attracted extensive attention for safety-critical control. However, due to the myopic one-step nature of CBF and the lack of principled methods to design the class-$\mathcal{K}$ functions, there are still fundamental limitations of current CBFs: optimality, stability, and feasibility. In this paper, we proposed a novel and unified approach to address these limitations with Adaptive Multi-step Control Barrier Function (AM-CBF), where we parameterize the class-$\mathcal{K}$ function by a neural network and train it together with the reinforcement learning policy. Moreover, to mitigate the myopic nature, we propose a novel \textit{multi-step training and single-step execution} paradigm to make CBF farsighted while the execution remains solving a single-step convex quadratic program. Our method is evaluated on the first and second-order systems in various scenarios, where our approach outperforms the conventional CBF both qualitatively and quantitatively.
It has been shown that satisfying state and control constraints while optimizing quadratic costs subject to desired (sets of) state convergence for affine control systems can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in this approach is ensuring the feasibility of these QPs, especially under tight control bounds and safety constraints of high relative degree. In this paper, we provide sufficient conditions for guranteed feasibility. The sufficient conditions are captured by a …
Carnegie Mellon University（卡内基梅隆大学）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
With the increasing need for safe control in the domain of autonomous driving, model-based safety-critical control approaches are widely used, especially Control Barrier Function (CBF)-based approaches. Among them, Exponential CBF (eCBF) is particularly popular due to its realistic applicability to high-relative-degree systems. However, for most of the optimization-based controllers utilizing CBF-based constraints, solution feasibility is a common issue arising from potential conflict among different constraints. Moreover, how to incorporate uncertainty into the eCBF-based constraints in high-…
Recent work showed that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs and observing state and control constraints can be reduced to quadratic programs (QP) by using control barrier functions (CBF) and control Lyapunov functions. In our own recent work, we defined high order CBFs (HOCBFs) to accommodating systems and constraints with arbitrary relative degrees, and a penalty method to increase the feasibility of the corresponding QPs. In this paper, we introduce adaptive CBF (AdaCBFs) that can accommodate time-varying control bounds and dynamics…
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与学院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）
Model information can be used to predict future trajectories, so it has huge potential to avoid dangerous region when implementing reinforcement learning (RL) on real-world tasks, like autonomous driving. However, existing studies mostly use model-free constrained RL, which causes inevitable constraint violations. This paper proposes a model-based feasibility enhancement technique of constrained RL, which enhances the feasibility of policy using generalized control barrier function (GCBF) defined on the distance to constraint boundary. By using the model information, the policy can be optimize…
Safe Learning Control with Optimality and Stability Guarantees
Shenzhen Key Lab for Advanced Motion Control and Modern Automation Equipments, School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, Guangdong 518055, China（深圳先进 motion 控制与现代自动化装备重点实验室，智能科学与工程学院，哈尔滨工业大学，深圳，广东 518055，中国）；School of Data Science, Lingnan University, Hong Kong（数据科学学院，岭南大学，香港）；School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（电气与电子工程学院，南洋理工大学，新加坡）；MIT CSAIL
We investigate the optimality of perturbation based algorithms in the stochastic and adversarial multi-armed bandit problems. For the stochastic case, we provide a unified regret analysis for both sub-Weibull and bounded perturbations when rewards are sub-Gaussian. Our bounds are instance optimal for sub-Weibull perturbations with parameter 2 that also have a matching lower tail bound, and all bounded support perturbations where there is sufficient probability mass at the extremes of the support. For the adversarial setting, we prove rigorous barriers against two natural solution approaches us…
Task-space Passivity-Based Control (PBC) for manipulation has numerous appealing properties, including robustness to modeling error and safety for human-robot interaction. Existing methods perform poorly in singular configurations, however, such as when all the robot's joints are fully extended. Additionally, standard methods for constrained task-space PBC guarantee passivity only when constraints are not active. We propose a convex-optimization-based control scheme that provides guarantees of singularity avoidance, passivity, and feasibility. This work paves the way for PBC with passivity gua…
Massachusetts Institute of Technology（麻省理工学院）；Boston University（波士顿大学）
Control Barrier Functions (CBFs) are becoming popular tools in guaranteeing safety for nonlinear systems and constraints, and they can reduce a constrained optimal control problem into a sequence of Quadratic Programs (QPs) for affine control systems. The recently proposed High Order Control Barrier Functions (HOCBFs) work for arbitrary relative degree constraints. One of the challenges in a HOCBF is to address the relative degree problem when a system has multiple control inputs, i.e., the relative degree could be defined with respect to different components of the control vector. This paper …
In this article, we present a control strategy for the problem of safe autonomous docking for a planar floating platform (Slider) that emulates the movement of a satellite. Employing the proposed strategy, Slider approaches a docking port with the right orientation, maintaining a safe distance, while always keeping a visual lock on the docking port throughout the docking maneuver. Control barrier functions are designed to impose the safety, direction of approach and visual locking constraints. Three control inputs of the Slider are shared among three barrier functions in enforcing the constrai…
Massachusetts Institute of Technology（麻省理工学院）；Boston University（波士顿大学）
It has been shown that optimizing quadratic costs while stabilizing affine control systems to desired (sets of) states subject to state and control constraints can be reduced to a sequence of Quadratic Programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). In this paper, we employ machine learning techniques to ensure the feasibility of these QPs, which is a challenging problem, especially for high relative degree constraints where High Order CBFs (HOCBFs) are required. To this end, we propose a sampling-based learning approach to learn a new feasibilit…
In this paper, we consider the problem of protecting a high-value area from being breached by sheep agents by crafting motions for dog robots. We use control barrier functions to pose constraints on the dogs' velocities that induce repulsion in the sheep relative to the high-value area. This paper extends the results developed in our prior work on the same topic in three ways. Firstly, we implement and validate our previously developed centralized herding algorithm on many robots. We show herding of up to five sheep agents using three dog robots. Secondly, as an extension to the centralized ap…
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Institude of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Proximal policy optimization(PPO) has been proposed as a first-order optimization method for reinforcement learning. We should notice that an exterior penalty method is used in it. Often, the minimizers of the exterior penalty functions approach feasibility only in the limits as the penalty parameter grows increasingly large. Therefore, it may result in the low level of sampling efficiency. This method, which we call proximal policy optimization with barrier method (PPO-B), keeps almost all advantageous spheres of PPO such as easy implementation and good generalization. Specifically, a new sur…
Feasibility-aware Learning of Robust Temporal Logic Controllers using BarrierNet
Amazon Robotics（亚马逊机器人）；Department of Mechanical Engineering（机械工程系）；Worcester Polytechnic Institute（沃斯堡技术学院）；Department of Electrical and Computer Engineering and the Department of Computer Science（电气与计算机工程系和计算机科学系）
Solving Max-Cut to Global Optimality via Feasibility-Preserving Graph Neural Networks
Davidson School of Chemical Engineering, Purdue University（普渡大学化学工程大卫逊学院）；Faculty of Computer Science, RWTH Aachen University（亚琛工业大学计算机科学学院）；Jacobs Technion-Cornell Institute, Cornell Tech（技术学院-康奈尔研究所）
University of Pennsylvania（宾夕法尼亚大学）；Google Brain Robotics（谷歌大脑机器人部门）
The need for robust control laws is especially important in safety-critical applications. We propose robust hybrid control barrier functions as a means to synthesize control laws that ensure robust safety. Based on this notion, we formulate an optimization problem for learning robust hybrid control barrier functions from data. We identify sufficient conditions on the data such that feasibility of the optimization problem ensures correctness of the learned robust hybrid control barrier functions. Our techniques allow us to safely expand the region of attraction of a compass gait walker that is …
Southwest University（西南大学）；University of Surrey（萨里大学）；University of York（约克大学）
We provide a novel approach to synthesize controllers for nonlinear continuous dynamical systems with control against safety properties. The controllers are based on neural networks (NNs). To certify the safety property we utilize barrier functions, which are represented by NNs as well. We train the controller-NN and barrier-NN simultaneously, achieving a verification-in-the-loop synthesis. We provide a prototype tool nncontroller with a number of case studies. The experiment results confirm the feasibility and efficacy of our approach.
KTH Royal Institute of Technology（瑞典皇家理工学院）；University of Pennsylvania（宾夕法尼亚大学）；Google Brain Robotics（谷歌大脑机器人）
Motivated by the lack of systematic tools to obtain safe control laws for hybrid systems, we propose an optimization-based framework for learning certifiably safe control laws from data. In particular, we assume a setting in which the system dynamics are known and in which data exhibiting safe system behavior is available. We propose hybrid control barrier functions for hybrid systems as a means to synthesize safe control inputs. Based on this notion, we present an optimization-based framework to learn such hybrid control barrier functions from data. Importantly, we identify sufficient conditi…
Optimal control problems with constraints ensuring safety and convergence to desired states can be mapped onto a sequence of real time optimization problems through the use of Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in these approaches is ensuring the feasibility of the resulting quadratic programs (QPs) if the system is affine in controls. The recently proposed penalty method has the potential to improve the existence of feasible solutions to such problems. In this paper, we further improve the feasibility robustness (i.e., feasibilit…
A Two-Phase Adaptive Balanced Penalty Method for Controllable Pareto Front Learning under Split Feasibility Conditions
Faculty of Applied Mathematics and Informatics, Hanoi University of Science and Technology（应用数学与信息技术学院，河内科学技术大学）；College of Engineering and Computer Science, VinUniversity（工程与计算机科学学院，Vin大学）
Language Generation: Complexity Barriers and Implications for Learning
Department of Computer Science, Pontifical Catholic University of Chile（智利天主教大学计算机科学系）；Institute for Mathematical and Computation Engineering, Pontifical Catholic University of Chile（智利天主教大学数学与计算工程研究所）；Faculty of Mathematics, Pontifical Catholic University of Chile（智利天主教大学数学学院）；National Center for Artificial Intelligence of Chile（智利人工智能国家中心）
