Low-Rank Extragradient Methods for Scalable Semidefinite Optimization
Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology(以色列理工学院数据与决策科学学院)
Dan Garber, Atara Kaplan
备注:
This version corrects an error in the previous version, as well as in the short version published in \textit{Operations Research Letters} \cite{garber2025low}: while in those versions we reported $\mathcal{O}(1/T)$ rates for the \textbf{best iterate}, in this corrected version these rates hold only w.r.t. the \textbf{average iterate}
We consider several classes of highly important semidefinite optimization problems that involve both a convex objective function (smooth or nonsmooth) and additional linear or nonlinear smooth and convex constraints, which are ubiquitous in statistics, machine learning, combinatorial optimization, and other domains. We focus on high-dimensional and plausible settings in which the problem admits a low-rank solution which also satisfies a low-rank complementarity condition. We provide several theoretical results proving that, under these circumstances, the well-known Extragradient method, when initialized in the proximity of an optimal primal-dual solution, converges to a solution of the constrained optimization problem with its standard convergence rates guarantees, using only low-rank singular value decompositions (SVD) to project onto the positive semidefinite cone, as opposed to computationally-prohibitive full-rank SVDs required in worst-case. Our approach is supported by numerical experiments conducted with a dataset of Max-Cut instances.
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
SGIT AI Lab, State Grid Corporation of China(国网SGIT人工智能实验室);School of Management, Xi’an Jiaotong University(西安交通大学管理学院);School of Software, Xi’an Jiaotong University(西安交通大学软件学院)
Noisy intermediate-scale quantum algorithm for semidefinite programming
Centre for Quantum Technologies, National University of Singapore(新加坡国立大学量子技术中心);Imperial College London(帝国理工学院);University of Oxford(牛津大学);MajuLab, CNRS-UNS-NUS-NTU International Joint Research Unit(Maju实验室,法国国家科学研究中心-尼斯大学-新加坡国立大学-南洋理工大学国际联合研究单位);National Institute of Education, Nanyang Technological University(南洋理工大学国立教育学院)
Fast, Scalable, Warm-Start Semidefinite Programming with Spectral Bundling and Sketching
Manning College of Information and Computer Sciences, University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校信息与计算机科学学院);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)
Split-Merge: A Difference-based Approach for Dominant Eigenvalue Problem
LMIB of the Ministry of Education, School of Mathematical Sciences, Beihang University(教育部离散数学与信息检索重点实验室,北京航空航天大学数学科学学院);National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University(工业智能与系统优化国家级前沿科学中心,东北大学)
Compact Lifted Relaxations for Low-Rank Optimization
Department of Analytics, Marketing and Operations, Imperial Business School(分析、营销与运营部,帝国商业学院);Management Science and Operations, London Business School(管理科学与运营,伦敦商业学院)
Scalable Second-order Riemannian Optimization for $K$-means Clustering
Department of Statistics, University of Illinois Urbana-Champaign(统计系,伊利诺伊大学厄巴纳-香槟分校);Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign(电气与计算机工程系,伊利诺伊大学厄巴纳-香槟分校);Department of Mathematics, University of Southern California(数学系,南加州大学)
Scalable Ground-State Certification of Quantum Spin Systems via Structured Noncommutative Polynomial Optimization
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);ICFO-Institut de Ciencies Fotoniques, The Barcelona Institute of Science and Technology(巴塞罗那科技研究所光科学中心);Laboratoire Kastler Brossel, Sorbonne Université, CNRS, ENS-PSL Research University, Collège de France(索邦大学卡斯特勒-布罗塞尔实验室);Inria Paris-Saclay(巴黎萨克雷国家信息与自动化研究所);LAAS-CNRS & Institute of Mathematics from Toulouse(图卢兹数学研究所);ICREA - Institució Catalana de Recerca i Estudis Avançats(加泰罗尼亚研究与高等教育局)