The error accumulation in the conjugate gradient method for degenerate problem
英文摘要：
This is a preprint of the article arXiv:2004.10242
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
相关论文
The error accumulation in the conjugate gradient method for degenerate problem
In this paper, we consider the conjugate gradient method for solving the problem of minimizing a quadratic function with additive noise in the gradient. Three concepts of noise were considered: antagonistic noise in the linear term, stochastic noise in the linear term, and noise in the quadratic term, as well as combinations of the first and second with the last. It was experimentally obtained that error accumulation is absent for any of the considered concepts, which differs from the folklore opinion that, as in accelerated methods, error accumulation must take place. The paper gives motivati…
math.OC
非凸有限极小极大问题的预处理三项共轭梯度框架
Preconditioned Three-Term Conjugate Gradient Framework for Nonconvex Finite Minimax Problems
National Center for Applied Mathematics in Chongqing, Chongqing Normal University（重庆应用数学中心，重庆师范大学）
本文提出一种基于双曲主导化的预处理三项共轭梯度框架，用于非凸有限极小极大优化，通过解析对称正定度量同时实现曲率吸收、预处理和线搜索，保证充分下降、全局收敛并收敛到 Clarke 稳定点。
math.OC
黎曼流形上退化非线性优化问题的稳定序列二次规划方法
A stabilized sequential quadratic programming method for degenerate nonlinear optimization problems on Riemannian manifolds
Graduate School of Management, Tokyo Metropolitan University（东京都立大学经营管理研究科）；Graduate School of Informatics, Kyoto University（京都大学情报学研究科）
本研究将稳定SQP框架扩展至黎曼流形，提出无约束规范假设下收敛的稳定SQP方法，经数值实验验证其对退化非线性优化问题的有效性。
math.OC
Smoothing SQP methods for solving degenerate nonsmooth constrained optimization problems with applications to bilevel programs
We consider a degenerate nonsmooth and nonconvex optimization problem for which the standard constraint qualification such as the generalized Mangasarian Fromovitz constraint qualification (GMFCQ) may not hold. We use smoothing functions with the gradient consistency property to approximate the nonsmooth functions and introduce a smoothing sequential quadratic programming (SQP) algorithm under the exact penalty framework. We show that any accumulation point of a selected subsequence of the iteration sequence generated by the smoothing SQP algorithm is a Clarke stationary point, provided that t…
math.OC
Accelerating Stochastic Gradient Descent For Least Squares Regression
Microsoft Research（微软研究院）；University of Washington（华盛顿大学）；Stanford University（斯坦福大学）
There is widespread sentiment that it is not possible to effectively utilize fast gradient methods (e.g. Nesterov's acceleration, conjugate gradient, heavy ball) for the purposes of stochastic optimization due to their instability and error accumulation, a notion made precise in d'Aspremont 2008 and Devolder, Glineur, and Nesterov 2014. This work considers these issues for the special case of stochastic approximation for the least squares regression problem, and our main result refutes the conventional wisdom by showing that acceleration can be made robust to statistical errors. In particular,…
cs.LGmath.OCmath.ST
