由于许多机器学习问题的层次结构，二层规划近年来变得越来越重要，然而，其内部问题和外部问题之间的复杂关联使得其求解极具挑战性。虽然基于自动微分的几种直观算法已经被提出并在一些应用中取得了成功，但对于双层模型的最优求解却没有给予太多的关注。是否存在一个更好的提法仍然是一个悬而未决的问题。在本文中，我们提出了一种改进的双层模型，收敛速度更快，更好地相比，目前的配方。我们提供了两个任务的理论保证和评估结果：数据超清洗和超表示学习。实证结果表明，我们的模型优于目前的双层模型有很大的差距。\emph{This is a concurrent work with \citet{liu2020generic} and we submitted to ICML 2020. Now we put it on the arxiv for record.}
英文摘要：
Due to the hierarchical structure of many machine learning problems, bilevel programming is becoming more and more important recently, however, the complicated correlation between the inner and outer problem makes it extremely challenging to solve. Although several intuitive algorithms based on the automatic differentiation have been proposed and obtained success in some applications, not much attention has been paid to finding the optimal formulation of the bilevel model. Whether there exists a better formulation is still an open problem. In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation. We provide theoretical guarantee and evaluation results over two tasks: Data Hyper-Cleaning and Hyper Representation Learning. The empirical results show that our model outperforms the current bilevel model with a great margin. \emph{This is a concurrent work with \citet{liu2020generic} and we submitted to ICML 2020. Now we put it on the arxiv for record.}
Distributionally Robust Control with End-to-End Statistically Guaranteed Metric Learning
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, PR China（自动化与智能感知学院，上海交通大学，中国）；Department of Mechanical Engineering, University of Victoria, Victoria, Canada（机械工程系，维多利亚大学，加拿大）
Stochastic Regret Guarantees for Online Zeroth- and First-Order Bilevel Optimization
Amirkabir University of Technology（阿姆斯泰尔大学）；University of Pennsylvania（宾夕法尼亚大学）；Samsung SDS Research America（三星SDS美国研究部）；University of California, Los Angeles（加州大学洛杉矶分校）
Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems
University of Waterloo（滑铁卢大学）；University of Victoria（维多利亚大学）；Southern University of Science and Technology（南方科技大学）；National Center for Applied Mathematics Shenzhen（深圳国家应用数学中心）
Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；Amazon（亚马逊）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）
提出 BILBO 算法，通过基于置信边界的可信集和单次查询策略同时优化双层黑盒问题，避免了重复下层优化，并在理论与实验上均验证了其有效性。
Sharper Analysis of Single-Loop Methods for Bilevel Optimization
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；School of Data Science, Fudan University（复旦大学数据科学学院）；SGIT AI Lab, State Grid Corporation of China（国家电网公司SGIT人工智能实验室）；Center for Intelligent Decision-Making and Machine Learning, School of Management, Xi’an Jiaotong University（西安交通大学管理学院智能决策与机器学习中心）
Bilevel Optimization for Feature Selection in the Data-Driven Newsvendor Problem
Technical University of Munich（慕尼黑工业大学）；Munich Data Science Institute（慕尼黑数据科学研究所）；Polytechnique Montréal（蒙特利尔理工大学）；Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）
Relax and penalize: a new bilevel approach to mixed-binary hyperparameter optimization
Massachusetts Institute of Technology（麻省理工学院）；University of Florence（佛罗伦萨大学）；Inria（法国国家信息与自动化研究所）；Trier University（特里尔大学）；University of Padova（帕多瓦大学）；Sapienza University of Rome（罗马大学）
Sequential QCQP for Bilevel Optimization with Line Search
Department of Electrical and Computer Engineering at Johns Hopkins University（约翰霍普金斯大学电气与计算机工程系）；Department of Systems and Industrial Engineering, The University of Arizona（亚利桑那大学系统与工业工程系）
Bilevel Learning with Inexact Stochastic Gradients
Department of Mathematical Sciences, University of Bath（巴斯大学数学科学系）；Department of Electronics & Electrical Communication Engineering, Indian Institute of Technology (IIT) Kharagpur（印度理工学院Kharagpur电子与电气通信工程系）；School of Mathematics and Statistics, University of Sydney（悉尼大学数学与统计学学院）
On the Complexity of Finding Stationary Points in Nonconvex Simple Bilevel Optimization
Department of Electrical and Computer Engineering, The University of Texas at Austin（电气与计算机工程系，德克萨斯大学奥斯汀分校）；Department of Systems and Industrial Engineering, The University of Arizona（系统与工业工程系，亚利桑那大学）
A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming
Amirkabir University of Technology（阿米尔卡比尔理工大学）；American University（美利坚大学）；University of Pennsylvania（宾夕法尼亚大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Towards a Unified Analysis of Neural Networks in Nonparametric Instrumental Variable Regression: Optimization and Generalization
University College London（伦敦大学学院）；Nanyang Technological University（南洋理工大学）；The University of Tokyo（东京大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所革新智能综合研究中心）
Bilevel Learning via Strategically Inexact Stochastic Gradient Descent
EPSRC Centre for Doctoral Training in Statistical Applied Mathematics at Bath（巴斯大学统计应用数学博士培训中心）；Australian Research Council（澳大利亚研究理事会）；Indian Institute of Technology (IIT), Kharagpur（印度理工学院（Kharagpur分校））；University of Melbourne（墨尔本大学）；Department of Electronics & Electrical Communication Engineering（电子与电气通信工程系）；School of Mathematics and Statistics（数学与统计学学院）
DUET: Decentralized Bilevel Optimization without Lower-Level Strong Convexity
Department of Electrical and Computer Engineering, The Ohio State University（俄亥俄州立大学电气与计算机工程系）；Department of Computer Science and Engineering, University of North Texas（北卡罗来纳州立大学计算机科学与工程系）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）
AsynDBT: Asynchronous Distributed Bilevel Tuning for efficient In-Context Learning with Large Language Models
Xinjiang Key Laboratory of Intelligent Computing and Smart Applications, School of Software, Xinjiang University（新疆智能计算与智能应用重点实验室，软件学院，新疆大学）；Department of Computer Science and Technology, Tongji University（计算机科学与技术系，同济大学）；School of Information Science and Engineering, Zaozhuang University（信息科学与工程学院，枣庄大学）；Xinjiang University, College of Geography and Remote Sensing Sciences（新疆大学，地理与遥感科学学院）；Hochschule Bielefeld-University of Applied Sciences and Arts（比勒菲尔德应用科学与艺术大学）；Xinjiang General Station of Exit and Entry Frontier Inspection（新疆出入境边防检查总站）
