While Variational Inequality (VI) is a well-established mathematical framework that subsumes Nash equilibrium and saddle-point problems, less is known about its extension, Quasi-Variational Inequalities (QVI). QVI allows for cases where the constraint set changes as the decision variable varies allowing for a more versatile setting. In this paper, we propose extra-gradient and gradient-based methods for solving a class of monotone Stochastic Quasi-Variational Inequalities (SQVI) and establish a rigorous convergence rate analysis for these methods. Our approach not only advances the theoretical understanding of SQVI but also demonstrates its practical applicability. Specifically, we highlight its effectiveness in reformulating and solving problems such as generalized Nash Equilibrium, bilevel optimization, and saddle-point problems with coupling constraints.
An adaptive analogue of the Yu. E. Nesterov method for variational inequalities with a strongly monotone operator is proposed. Some estimates are obtained for the parameters determining the quality of the solution of the variational inequality depending on the number of iterations.
It is known that for convex sets, the KKM condition is equivalent to the finite intersection property. We use this equivalence to obtain a characterisation of monotone operators in terms of convex KKM maps and in terms of the existence of solutions to Minty variational inequalities. The latter result provides a converse to the seminal theorem of Minty.
Basing on some recently proposed methods for solving variational inequalities with non-smooth operators, we propose an analogue of the Mirror Prox method for the corresponding class of problems under the assumption of relative smoothness and relative strong monotonicity of the operator.
The article is devoted to the development of numerical methods for solving variational inequalities with relatively strongly monotone operators. We consider two classes of variational inequalities related to some analogs of the Lipschitz condition of the operator that appeared several years ago. One of these classes is associated with the relative boundedness of the operator, and the other one with the analog of the Lipschitz condition (namely, relative smoothness). For variational inequalities with relatively bounded and relatively strongly monotone operators, we introduce a modification of t…
In this paper an inexact proximal point method for variational inequalities in Hadamard manifolds is introduced and studied its convergence properties. The main tool used for presenting the method is the concept of enlargement of monotone vector fields, which generalizes the concept of enlargement of monotone operators from the linear setting to the Riemannian context. As an application, an inexact proximal point method for constrained optimization problems is obtained.
We study nested variational inequalities, which are variational inequalities whose feasible set is the solution set of another variational inequality. We present a projected averaging Tikhonov algorithm requiring the weakest conditions in the literature to guarantee the convergence to solutions of the nested variational inequality. Specifically, we only need monotonicity of the upper- and the lower-level variational inequalities. Also, we provide the first complexity analysis for nested variational inequalities considering optimality of both the upper- and lower-level.
Tseng's forward-backward-forward algorithm is a valuable alternative for Korpelevich's extragradient method when solving variational inequalities over a convex and closed set governed by monotone and Lipschitz continuous operators, as it requires in every step only one projection operation. However, it is well-known that Korpelevich's method converges and can therefore be used also for solving variational inequalities governed by pseudo-monotone and Lipschitz continuous operators. In this paper, we first associate to a pseudo-monotone variational inequality a forward-backward-forward dynamical…
In this paper we present some new results on the existence of solutions of generalized variational inequalities in real reflexive Banach spaces with Fréchet differentiable norms. Moreover, we also give some theorems about the structure of solution sets. The results obtained in this paper improve and extend the ones announced by Fang and Peterson [1] to infinite dimensional spaces.
We examine a class of stochastic mirror descent dynamics in the context of monotone variational inequalities (including Nash equilibrium and saddle-point problems). The dynamics under study are formulated as a stochastic differential equation driven by a (single-valued) monotone operator and perturbed by a Brownian motion. The system's controllable parameters are two variable weight sequences that respectively pre- and post-multiply the driver of the process. By carefully tuning these parameters, we obtain global convergence in the ergodic sense, and we estimate the average rate of convergence…
Nonconvex-nonconcave saddle-point optimization in machine learning has triggered lots of research for studying non-monotone variational inequalities (VI). In this work, we introduce two mirror frameworks, called mirror extragradient method and mirror extrapolation method, for approximating solutions to relatively Lipschitz and monotone-like VIs. The former covers the well-known Nemirovski's mirror prox method and Nesterov's dual extrapolation method, and the recently proposed Bregman extragradient method; all of them can be reformulated into a scheme that is very similar to the original form o…
In this paper, we consider the monotone generalized variational inequality (MGVI) where the monotone operator is Lipschitz continuous. Inspired by the extragradient method and the projection contraction algorithms for monotone variational inequality (MVI), we propose a class of PGA-based Predictor-Corrector algorithms for MGVI. A significant characteristic of our algorithms for separable multi-blocks convex optimization problems is that they can be well adapted for parallel computation. Numerical simulations about different models for sparsity recovery show the wide applicability and effective…
In this paper, we study a class of generalized monotone variational inequality (GMVI) problems whose operators are not necessarily monotone (e.g., pseudo-monotone). We present non-Euclidean extragradient (N-EG) methods for computing approximate strong solutions of these problems, and demonstrate how their iteration complexities depend on the global Lipschitz or Hölder continuity properties for their operators and the smoothness properties for the distance generating function used in the N-EG algorithms. We also introduce a variant of this algorithm by incorporating a simple line-search procedu…
The standard algorithms for solving large-scale convex-concave saddle point problems, or, more generally, variational inequalities with monotone operators, are proximal type algorithms which at every iteration need to compute a prox-mapping, that is, to minimize over problem's domain $X$ the sum of a linear form and the specific convex distance-generating function underlying the algorithms in question. Relative computational simplicity of prox-mappings, which is the standard requirement when implementing proximal algorithms, clearly implies the possibility to equip $X$ with a relatively comput…
The majority of First Order methods for large-scale convex-concave saddle point problems and variational inequalities with monotone operators are proximal algorithms which at every iteration need to minimize over problem's domain X the sum of a linear form and a strongly convex function. To make such an algorithm practical, X should be proximal-friendly -- admit a strongly convex function with easy to minimize linear perturbations. As a byproduct, X admits a computationally cheap Linear Minimization Oracle (LMO) capable to minimize over X linear forms. There are, however, important situations …
In this paper, we propose a general extra-gradient scheme for solving monotone variational inequalities (VI), referred to here as Approximation-based Regularized Extra-gradient method (ARE). The first step of ARE solves a VI subproblem with an approximation operator satisfying a $p^{th}$-order Lipschitz bound with respect to the original mapping, further coupled with the gradient of a $(p+1)^{th}$-order regularization. The optimal global convergence is guaranteed by including an additional extra-gradient step, while a $p^{th}$-order superlinear local convergence is shown to hold if the VI is s…
In this paper, we introduce some adaptive methods for solving variational inequalities with relatively strongly monotone operators. Firstly, we focus on the modification of the recently proposed, in smooth case [1], adaptive numerical method for generalized smooth (with Hölder condition) saddle point problem, which has convergence rate estimates similar to accelerated methods. We provide the motivation for such an approach and obtain theoretical results of the proposed method. Our second focus is the adaptation of widespread recently proposed methods for solving variational inequalities with r…
A Variable-Metric Non-monotone Line Search Method for Mixed Variational Inequalities and Equilibrium Problems
King Fahd University of Petroleum & Minerals (KFUPM)（法赫德国王石油矿产大学）；Interdisciplinary Research Center (IRC) for Smart Mobility and Logistics, King Fahd University of Petroleum & Minerals (KFUPM)（法赫德国王石油矿产大学智能交通与物流交叉研究中心）
We introduce an inexact oracle model for variational inequalities (VI) with monotone operator, propose a numerical method which solves such VI's and analyze its convergence rate. As a particular case, we consider VI's with Hölder-continuous operator and show that our algorithm is universal. This means that without knowing the Hölder parameter $ν$ and Hölder constant $L_ν$ it has the best possible complexity for this class of VI's, namely our algorithm has complexity $O\left( \inf_{ν\in[0,1]}\left(\frac{L_ν}{\varepsilon} \right)^{\frac{2}{1+ν}}R^2 \right)$, where $R$ is the size of the feasible…
We consider a stochastic variational inequality (SVI) problem with a continuous and monotone mapping over a closed and convex set. In strongly monotone regimes, we present a variable sample-size averaging scheme (VS-Ave) that achieves a linear rate with an optimal oracle complexity. In addition, the iteration complexity is shown to display a muted dependence on the condition number compared with standard variance-reduced projection schemes. To contend with merely monotone maps, we develop amongst the first proximal-point algorithms with variable sample-sizes (PPAWSS), where increasingly accura…
We introduce and study the convergence properties of a projection-type algorithm for solving the variational inequality problem for point-to-set operators. No monotoni\-city assumption is used in our analysis. The operator defining the problem is only assumed to be continuous in the point-to-set sense, i.e., inner- and outer-semicontinuous. Additionally, we assume non-emptiness of the so-called dual solution set. We prove that the whole sequence of iterates converges to a solution of the variational inequality. Moreover, we provide numerical experiments illustrating the behavior of our iterate…
We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregulari…
This paper establishes several upper and lower estimates for the maximal number of the connected components of the solution sets of monotone affine vector variational inequalities. Our results give a partial solution to Question~2 in [N.D. Yen and J.-C. Yao, \textit{Monotone affine vector variational inequalities}, Optimization 60 (2011), pp. 53--68] and point out that the number depends not only on the number of the criteria but also on the number of variables of the vector variational inequality under investigation.
This paper is concerned with some new projection methods for solving variational inequality problems with monotone and Lipschitz-continuous mapping in Hilbert space. First, we propose the projected reflected gradient algorithm with a constant stepsize. It is similar to the projected gradient method, namely, the method requires only one projection onto the feasible set and only one value of the mapping per iteration. This distinguishes our method from most other projection-type methods for variational inequalities with monotone mapping. Also we prove that it has R-linear rate of convergence und…
The Popov's Algorithm with Optimal Bounded Stepsize for Generalized Monotone Variational Inequalities
Faculty of Applied Sciences, HCMC University of Technology and Engineering（胡志明市技术与工程学院应用科学学院）；Faculty of Basic Sciences, Van Lang University（文郎大学基础科学学院）；School of Mathematical Sciences, University of Southampton（南安普顿大学数学科学学院）
We show that the lower limit of a sequence of maximal monotone operators on a reflexive Banach space is a representable monotone operator. As a consequence, we obtain that the variational sum of maximal monotone operators and the variational composition of a maximal monotone operator with a linear continuous operator are both representable monotone operators.
The Past Extragradient (PEG) [Popov, 1980] method, also known as the Optimistic Gradient method, has known a recent gain in interest in the optimization community with the emergence of variational inequality formulations for machine learning. Recently, in the unconstrained case, Golowich et al. [2020] proved that a $O(1/N)$ last-iterate convergence rate in terms of the squared norm of the operator can be achieved for Lipschitz and monotone operators with a Lipschitz Jacobian. In this work, by introducing a novel analysis through potential functions, we show that (i) this $O(1/N)$ last-iterate …
An efficient proximal-gradient-based method, called proximal extrapolated gradient method, is designed for solving monotone variational inequality in Hilbert space. The proposed method extends the acceptable range of parameters to obtain larger step sizes. The step size is predicted based a local information of the operator and corrected by linesearch procedures to satisfy a very weak condition, which is even weaker than the boundedness of sequence generated and always holds when the operator is the gradient of a convex function. We establish its convergence and ergodic convergence rate in the…
In this paper, we propose a unifying framework incorporating several momentum-related search directions for solving strongly monotone variational inequalities. The specific combinations of the search directions in the framework are made to guarantee the optimal iteration complexity bound of $\mathcal{O}\left(κ\ln(1/ε)\right)$ to reach an $ε$-solution, where $κ$ is the condition number. This framework provides the flexibility for algorithm designers to train -- among different parameter combinations -- the one that best suits the structure of the problem class at hand. The proposed framework in…
In infinite-dimensional Hilbert spaces we device a class of strongly convergent primal-dual schemes for solving variational inequalities defined by a Lipschitz continuous and pseudomonote map. Our novel numerical scheme is based on Tseng's forward-backward-forward scheme, which is known to display weak convergence, unless very strong global monotonicity assumptions are made on the involved operators. We provide a simple augmentation of this algorithm which is computationally cheap and still guarantees strong convergence to a minimal norm solution of the underlying problem. We provide an adapti…
An adaptive proximal method for a special class of variational inequalities and related problems is proposed. For example, the so-called mixed variational inequalities and composite saddle problems are considered. Some estimates of the necessary number of iterations are obtained to achieve a given quality of the solution.
The aim of the paper is twofold. Firstly, by using the constant rank level set theorem from differential geometry, we establish sharp upper bounds for the dimensions of the solution sets of polynomial variational inequalities under mild conditions. Secondly, a classification of polynomial variational inequalities based on dimensions of their solution sets is introduced and investigated. Several illustrative examples are provided.
In this paper we propose new algorithms for solving a class of structured monotone variational inequality (VI) problems over compact feasible sets. By identifying the gradient components existing in the operator of VI, we show that it is possible to skip computations of the gradients from time to time, while still maintaining the optimal iteration complexity for solving these VI problems. Specifically, for deterministic VI problems involving the sum of the gradient of a smooth convex function $\nabla G$ and a monotone operator $H$, we propose a new algorithm, called the mirror-prox sliding met…
