发表机构
The University of Texas at Austin（德克萨斯大学奥斯汀分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种随机增广拉格朗日框架，用于求解非凸期望约束优化问题，通过随机二阶方法非精确求解子问题，建立了获得近似二阶稳定点的迭代与样本复杂度保证，并在两个机器学习任务上验证了性能。
AI 中文摘要
本文提出并分析了一种增广拉格朗日框架，用于求解在闭凸约束集上带有基于期望的等式约束的随机非凸优化问题。该框架生成一系列非凸原始子问题，并使用随机二阶方法进行非精确求解。我们在原始子问题解的精度满足相应条件的情况下，建立了在期望意义下和以指定概率获得近似二阶稳定点的迭代复杂度结果。进一步，在关于一般随机二阶子问题求解器的理论保证的合理假设下，我们建立了该框架的样本复杂度结果。此外，我们整合了三种现有的随机二阶求解器，并推导了获得近似二阶稳定点的相应确定性和概率性样本复杂度结果。据我们所知，在非凸期望约束优化中获得近似二阶稳定点的此类样本复杂度保证此前尚未建立。最后，我们在两个非凸机器学习问题上展示了所提出框架的实证性能。
英文摘要
In this paper, we propose and analyze an augmented Lagrangian framework for solving stochastic nonconvex optimization problems with expectation-based equality constraints over a closed and convex constraint set. The framework generates a sequence of nonconvex primal subproblems, which are solved inexactly using stochastic second-order methods. We establish iteration complexity results for obtaining approximate second-order stationary points, both in expectation and with prescribed probability, under corresponding conditions on the accuracy of the primal subproblem solutions. We further establish sample complexity results for the framework with general stochastic second-order subproblem solvers under reasonable assumptions on their theoretical guarantees. Moreover, we incorporate three existing stochastic second-order solvers and derive the corresponding deterministic and probabilistic sample complexity results for obtaining approximate second-order stationary points. To the best of our knowledge, such sample complexity guarantees for obtaining approximate second-order stationary points in nonconvex expectation-constrained optimization have not been established previously. Finally, we demonstrate the empirical performance of the proposed framework on two nonconvex machine learning problems.
Comments50 pages, 2 figures
