Stochastic bilevel optimization generalizes the classic stochastic optimization from the minimization of a single objective to the minimization of an objective function that depends the solution of another optimization problem. Recently, stochastic bilevel optimization is regaining popularity in emerging machine learning applications such as hyper-parameter optimization and model-agnostic meta learning. To solve this class of stochastic optimization problems, existing methods require either double-loop or two-timescale updates, which are sometimes less efficient. This paper develops a new optimization method for a class of stochastic bilevel problems that we term Single-Timescale stochAstic BiLevEl optimization (STABLE) method. STABLE runs in a single loop fashion, and uses a single-timescale update with a fixed batch size. To achieve an $ε$-stationary point of the bilevel problem, STABLE requires ${\cal O}(ε^{-2})$ samples in total; and to achieve an $ε$-optimal solution in the strongly convex case, STABLE requires ${\cal O}(ε^{-1})$ samples. To the best of our knowledge, this is the first bilevel optimization algorithm achieving the same order of sample complexity as the stochastic gradient descent method for the single-level stochastic optimization.
Stochastic bilevel optimization, which captures the inherent nested structure of machine learning problems, is gaining popularity in many recent applications. Existing works on bilevel optimization mostly consider either unconstrained problems or constrained upper-level problems. This paper considers the stochastic bilevel optimization problems with equality constraints both in the upper and lower levels. By leveraging the special structure of the equality constraints problem, the paper first presents an alternating implicit projected SGD approach and establishes the $\tilde{\cal O}(ε^{-2})$ s…
A Provably Convergent Plug-and-Play Framework for Stochastic Bilevel Optimization
Institute of Operations Research and Information Engineering, Beijing University of Technology（北京理工大学运筹学与信息工程学院）；National Center for Applied Mathematics Shenzhen（深圳应用数学中心）；Department of Mathematics, Southern University of Science and Technology（南方科技大学数学系）；School of Science, Beijing University of Posts and Telecommunications（北京邮电大学理学院）；Department of Mathematics, and National Center for Applied Mathematics Shenzhen, Southern University of Science and Technology（南方科技大学数学系和深圳应用数学中心）
SPABA: A Single-Loop and Probabilistic Stochastic Bilevel Algorithm Achieving Optimal Sample Complexity
National Center for Applied Mathematics Shenzhen, Southern University of Science and Technology, Shenzhen, China（深圳应用数学中心，南方科技大学，深圳，中国）；Institute of Operations Research and Information Engineering, Beijing University of Technology, Beijing 100124, P.R. China（运筹与信息工程研究所，北京理工大学，北京100124，中华人民共和国）；Department of Mathematics, Southern University of Science and Technology, Shenzhen, China（南方科技大学数学系，深圳，中国）
Temple University（天普大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；University of Florida（佛罗里达大学）
Bilevel optimization has been applied to a wide variety of machine learning models, and numerous stochastic bilevel optimization algorithms have been developed in recent years. However, most existing algorithms restrict their focus on the single-machine setting so that they are incapable of handling the distributed data. To address this issue, under the setting where all participants compose a network and perform peer-to-peer communication in this network, we developed two novel decentralized stochastic bilevel optimization algorithms based on the gradient tracking communication mechanism and …
The University of Iowa（爱荷华大学）；Nanjing University（南京大学）
In this paper, we consider non-convex stochastic bilevel optimization (SBO) problems that have many applications in machine learning. Although numerous studies have proposed stochastic algorithms for solving these problems, they are limited in two perspectives: (i) their sample complexities are high, which do not match the state-of-the-art result for non-convex stochastic optimization; (ii) their algorithms are tailored to problems with only one lower-level problem. When there are many lower-level problems, it could be prohibitive to process all these lower-level problems at each iteration. To…
Texas A&M University（得克萨斯农工大学）；University of Iowa（爱荷华大学）
In this paper, we study multi-block min-max bilevel optimization problems, where the upper level is non-convex strongly-concave minimax objective and the lower level is a strongly convex objective, and there are multiple blocks of dual variables and lower level problems. Due to the intertwined multi-block min-max bilevel structure, the computational cost at each iteration could be prohibitively high, especially with a large number of blocks. To tackle this challenge, we present a single-loop randomized stochastic algorithm, which requires updates for only a constant number of blocks at each it…
Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
Center for Data Science, Peking University（北京大学数据科学中心）；Center for Machine Learning Research, Peking University AI for Science Institute（北京大学人工智能科学研究院机器学习研究中心）
Bilevel optimization has become a powerful framework in various machine learning applications including meta-learning, hyperparameter optimization, and network architecture search. There are generally two classes of bilevel optimization formulations for machine learning: 1) problem-based bilevel optimization, whose inner-level problem is formulated as finding a minimizer of a given loss function; and 2) algorithm-based bilevel optimization, whose inner-level solution is an output of a fixed algorithm. For the first class, two popular types of gradient-based algorithms have been proposed for hy…
Bilevel optimization has arisen as a powerful tool for many machine learning problems such as meta-learning, hyperparameter optimization, and reinforcement learning. In this paper, we investigate the nonconvex-strongly-convex bilevel optimization problem. For deterministic bilevel optimization, we provide a comprehensive convergence rate analysis for two popular algorithms respectively based on approximate implicit differentiation (AID) and iterative differentiation (ITD). For the AID-based method, we orderwisely improve the previous convergence rate analysis due to a more practical parameter …
University of Michigan（密歇根大学）；University of California, Riverside（加州大学河滨分校）；University of British Columbia（不列颠哥伦比亚大学）
Standard federated optimization methods successfully apply to stochastic problems with single-level structure. However, many contemporary ML problems -- including adversarial robustness, hyperparameter tuning, and actor-critic -- fall under nested bilevel programming that subsumes minimax and compositional optimization. In this work, we propose \fedblo: A federated alternating stochastic gradient method to address general nested problems. We establish provable convergence rates for \fedblo in the presence of heterogeneous data and introduce variations for bilevel, minimax, and compositional op…
Bilevel Learning via Inexact Stochastic Gradient Descent
EPSRC Centre for Doctoral Training in Statistical Applied Mathematics at Bath（巴斯大学统计应用数学博士培训中心）；Australian Research Council（澳大利亚研究理事会）；Indian Institute of Technology (IIT), Kharagpur（印度理工学院（Kharagpur分校））；University of Melbourne（墨尔本大学）；Department of Electronics & Electrical Communication Engineering（电子与电气通信工程系）；School of Mathematics and Statistics（数学与统计学学院）
Fully First-Order Methods for Decentralized Bilevel Optimization
The Hong Kong University of Science and Technology（香港科技大学）；University of California Davis（加州大学戴维斯分校）；Rice University（莱斯大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The Ohio State University（俄亥俄州立大学）；Iowa State University（爱荷华州立大学）；University of Minnesota（明尼苏达大学）；IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
In recent years, decentralized bilevel optimization problems have received increasing attention in the networking and machine learning communities thanks to their versatility in modeling decentralized learning problems over peer-to-peer networks (e.g., multi-agent meta-learning, multi-agent reinforcement learning, personalized training, and Byzantine-resilient learning). However, for decentralized bilevel optimization over peer-to-peer networks with limited computation and communication capabilities, how to achieve low sample and communication complexities are two fundamental challenges that r…
Fine-grained Analysis of Stability and Generalization for Stochastic Bilevel Optimization
College of Informatics, Huazhong Agricultural University（华中农业大学信息学院）；Engineering Research Center of Intelligent Technology for Agriculture, Ministry of Education（教育部农业智能技术工程研究中心）；School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）
We analyse the asymptotic properties of a continuous-time, two-timescale stochastic approximation algorithm designed for stochastic bilevel optimisation problems in continuous-time models. We obtain the weak convergence rate of this algorithm in the form of a central limit theorem. We also demonstrate how this algorithm can be applied to several continuous-time bilevel optimisation problems.
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Pittsburgh（匹兹堡大学）
Bilevel optimization recently has attracted increased interest in machine learning due to its many applications such as hyper-parameter optimization and meta learning. Although many bilevel methods recently have been proposed, these methods do not consider using adaptive learning rates. It is well known that adaptive learning rates can accelerate optimization algorithms. To fill this gap, in the paper, we propose a novel fast adaptive bilevel framework to solve stochastic bilevel optimization problems that the outer problem is possibly nonconvex and the inner problem is strongly convex. Our fr…
In this paper, we establish the almost sure convergence of two-timescale stochastic gradient descent algorithms in continuous time under general noise and stability conditions, extending well known results in discrete time. We analyse algorithms with additive noise and those with non-additive noise. In the non-additive case, our analysis is carried out under the assumption that the noise is a continuous-time Markov process, controlled by the algorithm states. The algorithms we consider can be applied to a broad class of bilevel optimisation problems. We study one such problem in detail, namely…
University of Michigan（密歇根大学）；Princeton University（普林斯顿大学）；Facebook, Inc.（Facebook公司）；Northwestern University（西北大学）
Temporal-Difference (TD) learning with nonlinear smooth function approximation for policy evaluation has achieved great success in modern reinforcement learning. It is shown that such a problem can be reformulated as a stochastic nonconvex-strongly-concave optimization problem, which is challenging as naive stochastic gradient descent-ascent algorithm suffers from slow convergence. Existing approaches for this problem are based on two-timescale or double-loop stochastic gradient algorithms, which may also require sampling large-batch data. However, in practice, a single-timescale single-loop s…
We present a novel regularization approach to train neural networks that enjoys better generalization and test error than standard stochastic gradient descent. Our approach is based on the principles of cross-validation, where a validation set is used to limit the model overfitting. We formulate such principles as a bilevel optimization problem. This formulation allows us to define the optimization of a cost on the validation set subject to another optimization on the training set. The overfitting is controlled by introducing weights on each mini-batch in the training set and by choosing their…
One fundamental problem when solving inverse problems is how to find regularization parameters. This article considers solving this problem using data-driven bilevel optimization, i.e. we consider the adaptive learning of the regularization parameter from data by means of optimization. This approach can be interpreted as solving an empirical risk minimization problem, and we analyze its performance in the large data sample size limit for general nonlinear problems. We demonstrate how to implement our framework on linear inverse problems, where we can further show the inverse accuracy does not …
University of Minnesota（明尼苏达大学）；The Chinese University of Hong Kong（香港中文大学）；Northwestern University（西北大学）；Princeton University（普林斯顿大学）
This paper analyzes a two-timescale stochastic algorithm framework for bilevel optimization. Bilevel optimization is a class of problems which exhibit a two-level structure, and its goal is to minimize an outer objective function with variables which are constrained to be the optimal solution to an (inner) optimization problem. We consider the case when the inner problem is unconstrained and strongly convex, while the outer problem is constrained and has a smooth objective function. We propose a two-timescale stochastic approximation (TTSA) algorithm for tackling such a bilevel problem. In the…
Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）；Japan Science and Technology Agency（日本科学技术振兴机构）
Model extraction attacks have become serious issues for service providers using machine learning. We consider an adversarial setting to prevent model extraction under the assumption that attackers will make their best guess on the service provider's model using query accesses, and propose to build a surrogate model that significantly keeps away the predictions of the attacker's model from those of the true model. We formulate the problem as a non-convex constrained bilevel optimization problem and show that for kernel models, it can be transformed into a non-convex 1-quadratically constrained …
(Stochastic) bilevel optimization is a frequently encountered problem in machine learning with a wide range of applications such as meta-learning, hyper-parameter optimization, and reinforcement learning. Most of the existing studies on this problem only focused on analyzing the convergence or improving the convergence rate, while little effort has been devoted to understanding its generalization behaviors. In this paper, we conduct a thorough analysis on the generalization of first-order (gradient-based) methods for the bilevel optimization problem. We first establish a fundamental connection…
Bilevel optimization have gained growing interests, with numerous applications found in meta learning, minimax games, reinforcement learning, and nested composition optimization. This paper studies the problem of distributed bilevel optimization over a network where agents can only communicate with neighbors, including examples from multi-task, multi-agent learning and federated learning. In this paper, we propose a gossip-based distributed bilevel learning algorithm that allows networked agents to solve both the inner and outer optimization problems in a single timescale and share information…
Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications
School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；Department of Computational Applied Math and Operations Research, Rice University（莱斯大学计算应用数学与运筹学系）
A Tale of Two Problems: Multi-Task Bilevel Learning Meets Equality Constrained Multi-Objective Optimization
Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA（电子与计算机工程系，俄亥俄州立大学，哥伦布，OH，USA）；Meta Platforms, Inc., Menlo Park, CA, USA（Meta平台公司，Menlo Park，CA，USA）
