一种用于双层变分不等式问题的惯性迭代正则化外梯度方法

Bilevel Optimization based on Iterative Approximation of Multiple Mappings

A large number of application problems involve two levels of optimization, where one optimization task is nested inside the other. These problems are known as bilevel optimization problems and have been studied by both classical optimization community and evolutionary optimization community. Most of the solution procedures proposed until now are either computationally very expensive or applicable to only small classes of bilevel optimization problems adhering to mathematically simplifying assumptions. In this paper, we propose an evolutionary optimization method that tries to reduce the comput…

Bilevel Distributed Optimization in Directed Networks

Motivated by emerging applications in wireless sensor networks and large-scale data processing, we consider distributed optimization over directed networks where the agents communicate their information locally to their neighbors to cooperatively minimize a global cost function. We introduce a new unifying distributed constrained optimization model that is characterized as a bilevel optimization problem. This model captures a wide range of existing problems over directed networks including: (i) Distributed optimization with linear constraints; (ii) Distributed unconstrained nonstrongly convex …

Extragradient method with variance reduction for stochastic variational inequalities

We propose an extragradient method with stepsizes bounded away from zero for stochastic variational inequalities requiring only pseudo-monotonicity. We provide convergence and complexity analysis, allowing for an unbounded feasible set, unbounded operator, non-uniform variance of the oracle and, also, we do not require any regularization. Alongside the stochastic approximation procedure, we iteratively reduce the variance of the stochastic error. Our method attains the optimal oracle complexity $\mathcal{O}(1/ε^2)$ (up to a logarithmic term) and a faster rate $\mathcal{O}(1/K)$ in terms of the…

参数区域搜索：一种混合整数双层优化问题的原始启发式方法

Parametric Region Search: A Mixed-Integer Bilevel Optimization Problem Primal Heuristic

本文提出参数区域搜索（PRS），一种用于混合整数双层优化问题的新型原始启发式方法，通过迭代探索关键区域来快速找到高质量可行解，优于现有无导数元启发式方法。

Fast Policy Extragradient Methods for Competitive Games with Entropy Regularization

This paper investigates the problem of computing the equilibrium of competitive games, which is often modeled as a constrained saddle-point optimization problem with probability simplex constraints. Despite recent efforts in understanding the last-iterate convergence of extragradient methods in the unconstrained setting, the theoretical underpinnings of these methods in the constrained settings, especially those using multiplicative updates, remain highly inadequate, even when the objective function is bilinear. Motivated by the algorithmic role of entropy regularization in single-agent reinfo…

具有标准LMO的条件梯度方法用于随机简单双层优化

Conditional Gradient Methods with Standard LMO for Stochastic Simple Bilevel Optimization

本文提出了一种基于标准LMO的条件梯度方法，用于解决随机简单双层优化问题，通过非渐近收敛速率证明和实验验证，提升了外层和内层目标函数的优化效率。

Smooth Bilevel Programming for Sparse Regularization

Iteratively reweighted least square (IRLS) is a popular approach to solve sparsity-enforcing regression problems in machine learning. State of the art approaches are more efficient but typically rely on specific coordinate pruning schemes. In this work, we show how a surprisingly simple reparametrization of IRLS, coupled with a bilevel resolution (instead of an alternating scheme) is able to achieve top performances on a wide range of sparsity (such as Lasso, group Lasso and trace norm regularizations), regularization strength (including hard constraints), and design matrices (ranging from cor…