带经典乘子更新的固定罚项线性化增广拉格朗日方法
A Fixed-Penalty Linearized Augmented Lagrangian Method with Classical Multiplier Updates
浏览论文内容
中文总结 AI 辅助
该研究提出NR-LALM方法，用正则化高斯-牛顿型步替代增广拉格朗日方法的非线性原始子问题，经Lean 4形式化后，在高维确定性、随机优化问题上表现良好且复杂度可控。
中文摘要 AI 辅助
增广拉格朗日方法对非线性等式约束优化问题十分有效，但求解其非线性原始子问题的成本较高。针对具有确定性或随机目标函数的光滑非凸问题，我们提出了非线性残差线性化增广拉格朗日方法（NR-LALM），该方法用正则化高斯-牛顿型步替代上述子问题，同时保留基于非线性约束残差的经典乘子更新。所得步长由一个对称正定线性方程组计算得到，但线性化原始模型与非线性残差更新之间的不匹配会在乘子恒等式中产生二次约束线性化误差。我们证明，在局部正则性条件下，该误差可被控制；乘子的有界性和轨迹的局部化是推导得出的，而非假设的。采用与精度无关的固定参数，确定性NR-LALM在O(ε⁻²)次迭代和一阶预言评估中找到ε-近似的Karush-Kuhn-Tucker（KKT）对。对于随机目标函数，带有安全重启的投影随机路径积分微分估计器，在期望下需要O(ε⁻³)次随机梯度评估和O(ε⁻²)次约束及雅可比评估。紧性和Kurdyka-Lojasiewicz条件进一步使确定性原始对偶序列具有有限长度收敛性。可选的最小范数二阶校正将约束线性化误差从二阶降至四阶，且不改变复杂度阶数。所有理论结果均在Lean 4中形式化。数值实验验证了预测的误差阶数，并在高维确定性和随机问题上表现出良好性能。
英文摘要
Augmented Lagrangian methods are effective for nonlinear equality-constrained optimization, but solving their nonlinear primal subproblems can be expensive. For smooth nonconvex problems with deterministic or stochastic objectives, we propose a nonlinear-residual linearized augmented Lagrangian method (NR-LALM) that replaces this subproblem by a regularized Gauss-Newton-type step while retaining the classical multiplier update based on the nonlinear constraint residual. The resulting step is computed from one symmetric positive-definite linear system, but the mismatch between the linearized primal model and the nonlinear-residual update produces a quadratic constraint-linearization error in the multiplier identity. We show that this error can be controlled under local regularity; multiplier boundedness and trajectory localization are derived rather than assumed. With fixed, accuracy-independent parameters, deterministic NR-LALM finds an $\varepsilon$-approximate Karush-Kuhn-Tucker (KKT) pair in $O(\varepsilon^{-2})$ iterations and first-order oracle evaluations. For stochastic objectives, a projected stochastic path-integrated differential estimator with safeguarded restarts requires, in expectation, $O(\varepsilon^{-3})$ stochastic-gradient evaluations and $O(\varepsilon^{-2})$ constraint and Jacobian evaluations. Compactness and a Kurdyka-Lojasiewicz condition further yield finite-length convergence of the deterministic primal-dual sequence. An optional minimum-norm second-order correction reduces the constraint-linearization error from second to fourth order without changing the complexity orders. All theoretical results are formalized in Lean 4. Numerical experiments confirm the predicted error orders and show favorable performance on high-dimensional deterministic and stochastic problems.
