一种用于确定性与随机非凸优化的自适应增广拉格朗日方法
An Adaptive Augmented Lagrangian Method for Deterministic and Stochastic Nonconvex Optimization
AI总结：
本文提出自适应罚参数更新与完整对偶步长的不精确增广拉格朗日算法，适用于确定性、随机函数及约束的非凸优化，经CUTEst与机器学习随机问题实验验证其优势。
AI中文摘要：
我们提出一种不精确增广拉格朗日算法，用于求解非线性非凸优化问题。与大多数近期提出的具有最坏情况复杂度保证的增广拉格朗日方法不同，该方法采用自适应罚参数更新和完整对偶步长。我们证明，当函数和约束均为确定性、函数为随机且约束为确定性、或两者均为随机时，该方法能达到增广拉格朗日方法已知的最佳最坏情况复杂度结果（对数因子除外）。在CUTEst测试问题上的实验证实，在确定性场景下，该方法相比罚参数非自适应和/或对偶步长较短的增广拉格朗日方法具有实际优势；在机器学习的随机约束优化问题上的数值结果也验证了这些发现。
英文摘要：
We present an inexact Augmented Lagrangian algorithm for solving nonlinear, non-convex optimization problems. Unlike most recently proposed Augmented Lagrangian methods with worst-case complexity guarantees, we utilize adaptive penalty parameter updates and full dual stepsizes. We show that the method matches the best known worst-case complexity results for Augmented Lagrangian methods (up to logarithmic factors) when both the function and constraints are deterministic, when the function is stochastic and the constraints are deterministic, and when both are stochastic. Experiments on CUTEst test problems confirm the practical advantages of the proposed approach over Augmented Lagrangian methods with non-adaptive penalty parameters and/or short dual step sizes in the deterministic setting. Numerical results on stochastic constrained optimization problems in machine learning also confirm these findings.
