An adaptive controller with bounded l2-gain from disturbances to errors is derived for linear time-invariant systems with uncertain parameters restricted to a finite set. The gain bound refers to the closed loop system, including the non-linear learning procedure. As a result, robustness to unmodelled dynamics (possibly nonlinear and infinite-dimensional) follows from the small gain theorem. The approach is based on a new zero-sum dynamic game formulation, which optimizes the trade-off between exploration and exploitation. An explicit upper bound on the optimal value function is stated in terms of semi-definite programming and a corresponding simple formula for an adaptive controller achieving the upper bound is given. Once the uncertain parameters have been sufficiently estimated, the controller behaves like standard H-infinity optimal control.
For linear time-invariant systems with uncertain parameters belonging to a finite set, we present a purely deterministic approach to multiple-model estimation and propose an algorithm based on the minimax criterion using constrained quadratic programming. The estimator tends to learn the dynamics of the system, and once the uncertain parameters have been sufficiently estimated, the estimator behaves like a standard Kalman filter.
For linear time-invariant systems having a state matrix with uncertain sign, we formulate a minimax adaptive control problem as a zero sum dynamic game. Explicit expressions for the optimal value function and the optimal control law are given in terms of a Riccati equation. The optimal control law is adaptive in the sense the past data is used to estimate the uncertain sign for prediction of future dynamics. Once the sign has been estimated, the controller behaves like standard H-infinity optimal state feedback.
We consider the problem of impulse control minimax in finite horizon, when cost functions $(C(t,x,ξ)>0)$. We show existence of value function of the problem. Moreover, the value function is characterized as the unique viscosity solution of an Isaacs quasi-variational inequality. This problem is in relation with an application in mathematical finance.
We study here the impulse control minimax problem. We allow the cost functionals and dynamics to be unbounded and hence the value functions can possibly be unbounded. We prove that the value function of the problem is continuous. Moreover, the value function is characterized as the unique viscosity solution of an Isaacs quasi-variational inequality. This problem is in relation with an application in mathematical finance.
Adaptive algorithms like AdaGrad and AMSGrad are successful in nonconvex optimization owing to their parameter-agnostic ability -- requiring no a priori knowledge about problem-specific parameters nor tuning of learning rates. However, when it comes to nonconvex minimax optimization, direct extensions of such adaptive optimizers without proper time-scale separation may fail to work in practice. We provide such an example proving that the simple combination of Gradient Descent Ascent (GDA) with adaptive stepsizes can diverge if the primal-dual stepsize ratio is not carefully chosen; hence, a fo…
In this paper we construct an infinite horizon minimax state observer for a linear stationary differential-algebraic equation (DAE) with uncertain but bounded input and noisy output. We do not assume regularity or existence of a (unique) solution for any initial state of the DAE. Our approach is based on a generalization of Kalman's duality principle. The latter allows us to transform minimax state estimation problem into a dual control problem for the adjoint DAE: the state estimate in the original problem becomes the control input for the dual problem and the cost function of the latter is, …
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Pittsburgh（匹兹堡大学）
In the paper, we propose a class of faster adaptive Gradient Descent Ascent (GDA) methods for solving the nonconvex-strongly-concave minimax problems by using the unified adaptive matrices, which include almost all existing coordinate-wise and global adaptive learning rates. In particular, we provide an effective convergence analysis framework for our adaptive GDA methods. Specifically, we propose a fast Adaptive Gradient Descent Ascent (AdaGDA) method based on the basic momentum technique, which reaches a lower gradient complexity of $\tilde{O}(κ^4ε^{-4})$ for finding an $ε$-stationary point …
In this paper, we investigate a decentralized control problem with nested subsystems, which is a general model for one-directional communication amongst many subsystems. The noises in our dynamics are modelled as uncertain variables which take values in finite sets. The objective is to minimize a worst-case shared cost. We demonstrate how the prescription approach can simplify the information structure and derive a structural form for optimal control strategies. The structural form allows us to restrict attention to control strategies whose domains do not grow in size with time, and thus, this…
Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（信息产业部模式分析与机器智能重点实验室）；School of Automation Science and Engineering, Xi’an Jiaotong University（西安交通大学自动化科学与工程学院）
Stochastic optimal control usually requires an explicit dynamical model with probability distributions, which are difficult to obtain in practice. In this work, we consider the linear quadratic regulator (LQR) problem of unknown linear systems and adopt a Wasserstein penalty to address the distribution uncertainty of additive stochastic disturbances. By constructing an equivalent deterministic game of the penalized LQR problem, we propose a Q-learning method with convergence guarantees to learn an optimal minimax controller.
The problem of estimating the $\mathcal{H}_\infty$-norm of an LTI system from noisy input/output measurements has attracted recent attention as an alternative to parameter identification for bounding unmodeled dynamics in robust control. In this paper, we study lower bounds for $\mathcal{H}_\infty$-norm estimation under a query model where at each iteration the algorithm chooses a bounded input signal and receives the response of the chosen signal corrupted by white noise. We prove that when the underlying system is an FIR filter, $\mathcal{H}_\infty$-norm estimation is no more efficient than …
To address the issue of inaccurate distributions in practical stochastic systems, a minimax linear-quadratic control method is proposed using the Wasserstein metric. Our method aims to construct a control policy that is robust against errors in an empirical distribution of underlying uncertainty, by adopting an adversary that selects the worst-case distribution. The opponent receives a Wasserstein penalty proportional to the amount of deviation from the empirical distribution. A closed-form expression of the finite-horizon optimal policy pair is derived using a Riccati equation. The result is …
A minimax optimal control approach for robust neural ODEs
Technical University Munich (TUM)（慕尼黑工业大学（TUM））；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所（MDSI））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML））
A Minimax Linear Quadratic Gaussian Method for Antiwindup Control Synthesis
school of Engineering and Information Technology, University of New South Wales at the Australian Defence Force Academy（新南威尔士大学澳大利亚国防学院工程与信息科技学院）；department of Mechanical and Mechatronics Engineering, University of Waterloo（滑铁卢大学机械与机电工程系）
In this paper, we propose a minimax linear-quadratic control method to address the issue of inaccurate distribution information in practical stochastic systems. To construct a control policy that is robust against errors in an empirical distribution of uncertainty, our method is to adopt an adversary, which selects the worst-case distribution. To systematically adjust the conservativeness of our method, the opponent receives a penalty proportional to the amount, measured with the Wasserstein metric, of deviation from the empirical distribution. In the finite-horizon case, using a Riccati equat…
This paper presents local asymptotic minimax regret lower bounds for adaptive Linear Quadratic Regulators (LQR). We consider affinely parametrized $B$-matrices and known $A$-matrices and aim to understand when logarithmic regret is impossible even in the presence of structural side information. After defining the intrinsic notion of an uninformative optimal policy in terms of a singularity condition for Fisher information we obtain local minimax regret lower bounds for such uninformative instances of LQR by appealing to van Trees' inequality (Bayesian Cramér-Rao) and a representation of regret…
