We investigate the existence and uniqueness of (locally) absolutely continuous trajectories of a penalty term-based dynamical system associated to a constrained variational inequality expressed as a monotone inclusion problem. Relying on Lyapunov analysis and on the ergodic continuous version of the celebrated Opial Lemma we prove weak ergodic convergence of the orbits to a solution of the constrained variational inequality under investigation. If one of the operators involved satisfies stronger monotonicity properties, then strong convergence of the trajectories can be shown.
We consider the minimization of a convex objective function subject to the set of minima of another convex function, under the assumption that both functions are twice continuously differentiable. We approach this optimization problem from a continuous perspective by means of a second order dynamical system with Hessian-driven damping and a penalty term corresponding to the constrained function. By constructing appropriate energy functionals, we prove weak convergence of the trajectories generated by this differential equation to a minimizer of the optimization problem as well as convergence f…
This paper proposes penalty schemes for a class of weakly coupled systems of Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. We show that the solutions of the penalized equations converge monotonically to those of the HJBQVIs. We further establish that the schemes are half-order accurate for HJBQVIs with Lipschitz coefficients, and first-order accurate for equations with more regular coefficients. Moreover, we construct the action regions and optimal impulse co…
We consider an optimization problem with strongly convex objective and linear inequalities constraints. To be able to deal with a large number of constraints we provide a penalty reformulation of the problem. As penalty functions we use a version of the one-sided Huber losses. The smoothness properties of these functions allow us to choose time-varying penalty parameters in such a way that the incremental procedure with the diminishing step-size converges to the exact solution with the rate $O(1/{\sqrt k})$. To the best of our knowledge, we present the first result on the convergence rate for …
We investigate geometric features of the unit ball corresponding to the sum of the nuclear norm of a matrix and the $l_1$ norm of its entries --- a common penalty function encouraging joint low rank and high sparsity. As a byproduct of this effort, we develop a calculus (or algebra) of faces for general convex functions, yielding a simple and unified approach for deriving inequalities balancing the various features of the optimization problem at hand, at the extreme points of the solution set.
A class of non-smooth and non-convex optimization problems with penalty constraints linked to variational inequalities (VI) is studied with respect to its shape differentiability. The specific problem stemming from quasi-brittle fracture describes an elastic body with a Barenblatt cohesive crack under the inequality condition of non-penetration at the crack faces. Based on the Lagrange approach and using smooth penalization with the Lavrentiev regularization, a formula for the shape derivative is derived. The explicit formula contains both primal and adjoint states and is useful for finding de…
Nonlinear constrained optimization problems are encountered in many scientific fields. To utilize the huge calculation power of current computers, many mathematic models are also rebuilt as optimization problems. Most of them have constrained conditions which need to be handled. Borrowing biological concepts, a study is accomplished for dealing with the constraints in the synthesis of a four-bar mechanism. Biologically regarding the constrained condition as a form of selection for characteristics of a population, four new algorithms are proposed, and a new explanation is given for the penalty …
We investigate the asymptotic convergence of the trajectories generated by the second order dynamical system $\ddot x(t) + γ\dot x(t) + \nabla ϕ(x(t))+β(t)\nabla ψ(x(t))=0$, where $ϕ,ψ:{\cal H}\rightarrow \R$ are convex and smooth functions defined on a real Hilbert space ${\cal H}$, $γ>0$ and $β$ is a function of time which controls the penalty term. We show weak convergence of the trajectories to a minimizer of the function $ϕ$ over the (nonempty) set of minima of $ψ$ as well as convergence for the objective function values along the trajectories, provided a condition expressed via the Fench…
In connection with the optimization problem $$\inf_{x\in argmin Ψ}\{Φ(x)+Θ(x)\},$$ where $Φ$ is a proper, convex and lower semicontinuous function and $Θ$ and $Ψ$ are convex and smooth functions defined on a real Hilbert space, we investigate the asymptotic behavior of the trajectories of the nonautonomous Levenberg-Marquardt dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partialΦ(x(t))\\ λ(t)\dot x(t) + \dot v(t) + v(t) + \nabla Θ(x(t))+β(t)\nabla Ψ(x(t))=0, \end{array}\right.\end{equation*} where $λ$ and $β$ are functions of time controlling the velocity and the penalty …
In this work, we consider a constrained convex problem with linear inequalities and provide an inexact penalty re-formulation of the problem. The novelty is in the choice of the penalty functions, which are smooth and can induce a non-zero penalty over some points in feasible region of the original constrained problem. The resulting unconstrained penalized problem is parametrized by two penalty parameters which control the slope and the curvature of the penalty function. With a suitable selection of these penalty parameters, we show that the solutions of the resulting penalized unconstrained p…
This paper considers variational inequalities (VI) defined by the conditional value-at-risk (CVaR) of uncertain functions and provides three stochastic approximation schemes to solve them. All methods use an empirical estimate of the CVaR at each iteration. The first algorithm constrains the iterates to the feasible set using projection. To overcome the computational burden of projections, the second one handles inequality and equality constraints defining the feasible set differently. Particularly, projection onto to the affine subspace defined by the equality constraints is achieved by matri…
We present a novel penalty approach for a class of quasi-variational inequalities (QVIs) involving monotone systems and interconnected obstacles. We show that for any given positive switching cost, the solutions of the penalized equations converge monotonically to those of the QVIs. We estimate the penalization errors and are able to deduce that the optimal switching regions are constructed exactly. We further demonstrate that as the switching cost tends to zero, the QVI degenerates into an equation of HJB type, which is approximated by the penalized equation at the same order (up to a log fac…
Stochastic variance reduced extragradient methods for solving hierarchical variational inequalities
Weierstrass Institute for Applied Analysis and Stochastics（魏尔斯特拉斯应用分析与随机学研究所）；Mannheim University（曼海姆大学）；Technion - Israel Institute of Technology（以色列理工学院）
It is well known that general variational inequalities provide us with a unified, natural, novel and simple framework to study a wide class of unrelated problems, which arise in pure and applied sciences. In this paper, we present a number of new and known numerical techniques for solving general variational inequalities and equilibrium problems using various techniques including projection, Wiener-Hopf equations, dynamical systems, auxiliary principle and penalty function. General variational-like inequalities are introduced and investigated. Properties of higher order strongly general convex…
We consider an optimal control problem $\cQ$ governed by an elliptic quasivariational inequality with unilateral constraints. The existence of optimal pairs of the problem is a well known result, see \cite{SS}, for instance. We associate to $\cQ$ a new optimal control problem $\wQ$, obtained by perturbing the state inequality (including the set of constraints and the nonlinear operator) and the cost functional, as well. Then, we provide sufficient conditions which guarantee the convergence of solutions of Problem $\wQ$ to a solution of Problem $\cQ$. The proofs are based on convergence results…
This paper discusses a special kind of convex constrained optimization problem, whose constraints consist of box inequalities and linear equalities. For this problem, in addition to general optimization algorithms such as exact penalty algorithm and interior point algorithm, there is a simple iterative algorithm that is simple to implement, which is favored by machine learning practitioners.
Asymptotic behaviour of coupled random dynamical systems with multiscale aspects
Institut für Numerische und Angewandte Mathematik, Universität Göttingen（哥廷根大学应用数学与数值计算研究所）；Mannheim University, Department of Mathematics（曼海姆大学数学系）；Vienna University of Economics and Business（维也纳经济与商业大学）；Rijksuniversiteit Groningen, Faculty of Science and Engineering, Systems, Control and Optimization — Bernoulli Institute（格罗宁根大学科学与工程学院系统、控制与优化伯努利研究所）
This paper proposes an efficient adaptive variant of a quadratic penalty accelerated inexact proximal point (QP-AIPP) method proposed earlier by the authors. Both the QP-AIPP method and its variant solve linearly set constrained nonconvex composite optimization problems using a quadratic penalty approach where the generated penalized subproblems are solved by a variant of the underlying AIPP method. The variant, in turn, solves a given penalized subproblem by generating a sequence of proximal subproblems which are then solved by an accelerated composite gradient algorithm. The main difference …
This paper deals with a general form of variational problems in Banach spaces which encompasses variational inequalities as well as minimization problems. We prove a characterization of local error bounds for the distance to the (primal-dual) solution set and give a sufficient condition for such an error bound to hold. In the second part of the paper, we consider an algorithm of augmented Lagrangian type for the solution of such variational problems. We give some global convergence properties of the method and then use the error bound theory to provide estimates for the rate of convergence and…
