This paper studies the extremum seeking control (ESC) problem for a class of constrained nonlinear systems. Specifically, we focus on a family of constraints allowing to reformulate the original nonlinear system in the so-called input-output normal form. To steer the system to optimize a performance function without knowing its explicit form, we propose a novel numerical optimization-based extremum seeking control (NOESC) design consisting of a constrained numerical optimization method and an inversion based feedforward controller. In particular, a projected gradient descent algorithm is exploited to produce the state sequence to optimize the performance function, whereas a suitable boundary value problem accommodates the finite-time state transition between each two consecutive points of the state sequence. Compared to available NOESC methods, the proposed approach i) can explicitly deal with output constraints; ii) the performance function can consider a direct dependence on the states of the internal dynamics; iii) the internal dynamics do not have to be necessarily stable. The effectiveness of the proposed ESC scheme is shown through extensive numerical simulations.
The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the global minimiser to be semi-global practically stable despite the presence of local saddle points. To achieve these results, we analyse the average system associated with the extremum seeking scheme via arguments based on the Fourier series.
In this paper we consider the problem of finding a Nash equilibrium (NE) via zeroth-order feedback information in games with merely monotone pseudogradient mapping. Based on hybrid system theory, we propose a novel extremum seeking algorithm which converges to the set of Nash equilibria in a semi-global practical sense. Finally, we present two simulation examples. The first shows that the standard extremum seeking algorithm fails, while ours succeeds in reaching NE. In the second, we simulate an allocation problem with fixed demand.
Event-Triggered Newton Extremum Seeking for Multivariable Optimization
State University of Rio de Janeiro (UERJ)（里约热内卢州立大学）；University of California at San Diego (UCSD)（加利福尼亚大学圣地亚哥分校）；University of California at Los Angeles (UCLA)（加利福尼亚大学洛杉矶分校）
This paper proposes a multivariable extremum seeking scheme using Fast Fourier Transform (FFT) for a network of subsystems working towards optimizing the sum of their local objectives, where the overall objective is the only available measurement. Here, the different inputs are perturbed with different dither frequencies, and the power spectrum of the overall output signal obtained using FFT is used to estimate the steady-state cost gradient w.r.t. each input. The inputs for the subsystems are then updated using integral control in order to drive the respective gradients to zero. This paper pr…
In this paper, we consider the problem of learning a generalized Nash equilibrium (GNE) in strongly monotone games. First, we propose a novel continuous-time solution algorithm that uses regular projections and first-order information. As second main contribution, we design a data-driven variant of the former algorithm where each agent estimates their individual pseudo-gradient via zero-order information, namely, measurements of their individual cost function values, as typical of extremum seeking control. Third, we generalize our setup and results for multi-agent systems with nonlinear dynami…
In this paper, we propose the concept of accelerated convergence that has originally been developed to speed up the convergence of numerical methods for extremum seeking (ES) loops. We demonstrate how the dynamics of ES loops may be analyzed to extract structural information about the generated output of the loop. This information is then used to distil the limit of the loop without having to wait for the system to converge to it.
This paper presents an extremum seeking control algorithm with an adaptive step-size that adjusts the aggressiveness of the controller based on the quality of the gradient estimate. The adaptive step-size ensures that the integral-action produced by the gradient descent does not destabilize the closed-loop system. To quantify the quality of the gradient estimate, we present a batch least squares estimator with a novel weighting and show that it produces bounded estimation errors, where the uncertainty is due to the curvature of the unknown cost function. The adaptive step-size then maximizes t…
In this paper, we describe a broad class of control functions for extremum seeking problems. We show that it unifies and generalizes existing extremum seeking strategies which are based on Lie bracket approximations, and allows to design new controls with favorable properties in extremum seeking and vibrational stabilization tasks. The second result of this paper is a novel approach for studying the asymptotic behavior of extremum seeking systems. It provides a constructive procedure for defining frequencies of control functions to ensure the practical asymptotic and exponential stability. In …
In this paper, we solve the problem of learning a generalized Nash equilibrium (GNE) in merely monotone games. First, we propose a novel continuous semi-decentralized solution algorithm without projections that uses first-order information to compute a GNE with a central coordinator. As the second main contribution, we design a gain adaptation scheme for the previous algorithm in order to alleviate the problem of improper scaling of the cost functions versus the constraints. Third, we propose a data-driven variant of the former algorithm, where each agent estimates their individual pseudogradi…
The paper presents some characteristics of unsteady scalar planar fields and studies their properties that are relevant to research on environmental extremum seeking and tracking environmental boundaries by mobile robots.
In recent years, an approach to extremum seeking control made it possible to design control vector fields that lead to asymptotic stability of the minimum point provided that the minimum value of the function is known a priori. In this work, we aim to relax that assumption. We propose an extremum-seeking control law that converges to the minimum point with vanishing control oscillations, without access to the minimum value of the cost function. We provide a numerical example to support our results.
Slow and fast dynamics in measure functional differential equations with state-dependent delays through averaging principles and applications to extremum seeking
Extremum seeking systems are powerful methods able to steer the input of a (dynamical) cost function towards an optimizer, without any prior knowledge of the cost function. To achieve their objective, they typically combine time-periodic signals with the on-line measurement of the cost. However, in some practical applications, the cost can only be measured during some regular time-intervals, and not continuously, contravening the classical extremum seeking framework. In this paper, we first analyze how existing Lie-bracket based extremum seeking systems behave when being fed with intermittent …
This paper formulates the problem of Extremum Seeking for optimization of cost functions defined on Riemannian manifolds. We extend the conventional extremum seeking algorithms for optimization problems in Euclidean spaces to optimization of cost functions defined on smooth Riemannian manifolds. This problem falls within the category of online optimization methods. We introduce the notion of geodesic dithers which is a perturbation of the optimizing trajectory in the tangent bundle of the ambient state manifolds and obtain the extremum seeking closed loop as a perturbation of the averaged grad…
We introduce a new class of extremum seeking controllers able to achieve fixed time convergence to the solution of optimization problems defined by static and dynamical systems. Unlike existing approaches in the literature, the convergence time of the proposed algorithms does not depend on the initial conditions and it can be prescribed a priori by tuning the parameters of the controller. Specifically, our first contribution is a novel gradient-based extremum seeking algorithm for cost functions that satisfy the Polyak-Lojasiewicz (PL) inequality with some coefficient κ> 0, and for which the e…
In this paper, we propose and practically evaluate a class of gradient-free control functions ensuring the motion of a unicycle-type system towards the extremum point of a time-varying cost function. We prove that the unicycle is able to track the extremum point, and illustrate our results by numerical simulations and experiments that show that the proposed control functions exhibit an improved tracking performance in comparison to standard extremum seeking laws based on Lie bracket approximations.
The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the global minimiser to be semi-global practically stable despite the presence of local minima. Under the assumption of a globally Lipschitz cost function, it is shown that the scheme, improved through a high-pass filter, makes the global minimiser practically stable with a global domain of attraction.
In this paper, a distributed non-model based seeking algorithm which combines the extremum seeking control (ESC) jointly with learning algorithms is proposed to seek a generalized Nash equilibrium (GNE) for a class of noncooperative games with coupled equality constraint. The strategy of each agent is restricted by both the coupled inter-agent constraint and local inequality constraints. Thanks to the ESC, it is unnecessary to know the specific expressions of agents' cost functions and local constraints and to know the strategies of other agents for the implementation of the proposed GNE seeki…
In this paper, we develop an extremum seeking control method integrated with iterative learning control to track a time-varying optimizer within finite time. The behavior of the extremum seeking system is analyzed via an approximating system - the modified Lie bracket system. The modified Lie bracket system is essentially an online integral-type iterative learning control law. The paper contributes to two fields, namely, iterative learning control and extremum seeking. First, an online integral type iterative learning control with a forgetting factor is proposed. Its convergence is analyzed vi…
We investigate stochastic averaging theory for locally Lipschitz discrete-time nonlinear systems with stochastic perturbation and its applications to convergence analysis of discrete-time stochastic extremum seeking algorithms. Firstly, by defining two average systems (one is continuous time, the other is discrete time), we develop discrete-time stochastic averaging theorem for locally Lipschitz nonlinear systems with stochastic perturbation. Our results only need some simple and applicable conditions, which are easy to verify, and remove a significant restriction present in existing results: …
A Model-Free Extremum Seeking Controller with Application to Tracking a Nonlinear Chemical Reaction
Max Planck Institute for Dynamics of Complex Technical Systems（马克斯·普朗克复杂技术系统动力学研究所）；Institute of Applied Mathematics and Mechanics, National Academy of Sciences of Ukraine（乌克兰国家科学院应用数学与力学研究所）；University of Klagenfurt（克拉根福大学）
The paper deals with the extremum seeking problem for a class of cost functions depending only on a part of state variables of a control system. This problem is related to the concept of partial asymptotic stability and analyzed by Lyapunov's direct method and averaging schemes. Sufficient conditions for the practical partial stability of a system with oscillating inputs are derived with the use of Lie bracket approximation techniques. These conditions are exploited to describe a broad class of extremum-seeking controllers ensuring the partial stability of the set of minima of a cost function.…
