King Abdullah University of Science and Technology (KAUST)(沙特阿拉伯国王阿卜杜拉科技大学(KAUST));School of Mathematics, University of Edinburgh(爱丁堡大学数学学院);Moscow Institute of Physics and Technology(莫斯科物理技术学院)
Robust principal component analysis (RPCA) is a well-studied problem with the goal of decomposing a matrix into the sum of low-rank and sparse components. In this paper, we propose a nonconvex feasibility reformulation of RPCA problem and apply an alternating projection method to solve it. To the best of our knowledge, we are the first to propose a method that solves RPCA problem without considering any objective function, convex relaxation, or surrogate convex constraints. We demonstrate through extensive numerical experiments on a variety of applications, including shadow removal, background estimation, face detection, and galaxy evolution, that our approach matches and often significantly outperforms current state-of-the-art in various ways.
An Alternating Manifold Proximal Gradient Method for Sparse PCA and Sparse CCA
The Chinese University of Hong Kong(香港中文大学);University of California, Davis(加利福尼亚大学戴维斯分校);The Pennsylvania State University(宾夕法尼亚州立大学);University of Minnesota(明尼苏达大学)
Katyusha X: Practical Momentum Method for Stochastic Sum-of-Nonconvex Optimization
Microsoft Research AI(微软研究院AI)
最小化非凸函数之和的问题(即,凸函数(即非凸函数的平均值)在机器学习中变得越来越重要,并且是PCA、SVD、正则化牛顿法、加速非凸优化等的核心机制。我们展示了如何可证明地获得一个加速的随机算法最小化的总和的非凸函数,由$\textit{adding one additional line}$著名的SVRG方法。这条线对应于动量,并显示了如何直接将动量应用于非凸函数之和的有限和随机最小化。作为一个副作用,我们的方法享有线性并行加速使用小批量。
A Quadratically Convergent Alternating Projection Method for Nonconvex Sets
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Institute of Applied Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院应用数学研究所);National University of Singapore(新加坡国立大学);University of Chicago(芝加哥大学)
A Riemannian Alternating Descent Ascent Algorithmic Framework for Nonconvex-Linear Minimax Problems on Riemannian Manifolds
ICMSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学);Ministry of Education Key Laboratory of NSLSCS, School of Mathematical Sciences, Nanjing Normal University(南京师范大学数学科学学院);Ministry of Education Key Laboratory of Mathematics and Information Networks, School of Mathematical Sciences, Beijing University of Posts and Telecommunications(北京邮电大学数学科学学院);Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong(香港中文大学系统工程与工程管理学系)
Derivative-free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problems
Shanghai University(上海大学);Newtouch Center for Mathematics of Shanghai University(上海大学新触数学中心);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)
Accelerated Alternating Projections for Robust Principal Component Analysis
Department of Mathematics, University of California, Los Angeles(加州大学洛杉矶分校数学系);Department of Mathematics, Hong Kong University of Science and Technology(香港理工大学数学系);School of Data Science, Fudan University(复旦大学数据科学学院)
Constrained Stochastic Nonconvex Optimization with State-dependent Markov Data
Halıcıoğlu Data Science Institute, University of California, San Diego(加州大学圣地亚哥分校哈利哲奥卢数据科学研究所);University of California, Davis(加州大学戴维斯分校);University of Waterloo(滑铁卢大学)
Towards An Efficient Approach for the Nonconvex $\ell_p$ Ball Projection: Algorithm and Analysis
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Center for Applied Mathematics of Henan Province, Henan University(河南大学河南省应用数学中心);Department of Mathematics, University of Washington(华盛顿大学数学系)
Zeroth-Order primal-dual Alternating Projection Gradient Algorithms for Nonconvex Minimax Problems with Coupled linear Constraints
Department of Mathematics, College of Sciences, Shanghai University(上海大学数学系);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);Newtouch Center for Mathematics of Shanghai University(上海大学数学中心);State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院数学科学国家重点实验室);School of Mathematical Sciences, University of Chinese Academy of Sciences(中国科学院大学数学科学学院)
On the efficient computation of proximal operators of affine-constrained nonconvex functions
National University of Singapore(新加坡国立大学);The University of Chicago(芝加哥大学);Institute of Operations Research and Analytics, National University of Singapore(新加坡国立大学运筹与分析研究所);Institute of Applied Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院应用数学研究所)
StablePCA: Distributionally Robust Learning of Shared Representations from Multi-Source Data
Rutgers University(罗杰斯大学);Peking University Health Science Center(北京大学医学部);Beijing International Center for Mathematical Research(北京国际数学研究中心);Inria École Normale Supérieure PSL Research University(Inria 常规大学);Zhejiang University(浙江大学)
Proximal Gradient Descent Ascent Methods for Nonsmooth Nonconvex-Concave Minimax Problems on Riemannian Manifolds
School of Mathematics, Sun Yat-sen University(中山大学数学学院);School of Computer Science and Engineering, Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University(中山大学计算机科学与工程学院)