具有非精确求值和复杂度保证的非凸最小化随机算法
A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees
浏览论文内容
中文总结 AI 辅助
本文研究光滑非凸函数的最小化问题，提出一种基于非精确梯度和Hessian访问的随机算法，通过等概率选择负曲率方向实现近似二阶最优性，并提供了期望界和高概率界的收敛性分析，在经验风险最小化中改进了梯度样本复杂度。
中文摘要 AI 辅助
我们考虑光滑非凸函数的最小化问题，通过梯度与Hessian的非精确oracle访问（不假设能获取函数值）来实现近似二阶最优性。我们方法的一个新特征是：如果选择近似负曲率方向作为步长，我们以等概率选择其正负方向。我们允许梯度在相对意义上是非精确的，并放宽了一阶与二阶最优性条件之间非精确阈值的耦合。我们的收敛性分析包括基于鞅分析的期望界和基于集中不等式的高概率界。我们将该算法应用于经验风险最小化问题，相比现有工作获得了改进的梯度样本复杂度。
英文摘要
We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen as the step, we choose its sense to be positive or negative with equal probability. We allow gradients to be inexact in a relative sense and relax the coupling between inexactness thresholds for the first- and second-order optimality conditions. Our convergence analysis includes both an expectation bound based on martingale analysis and a high-probability bound based on concentration inequalities. We apply our algorithm to empirical risk minimization problems and obtain improved gradient sample complexity over existing works.
发表机构
- University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
机构由 AI 辅助整理，请以论文原文为准。
