发表机构
Nanjing University of Science and Technology; Nanjing University; Zhejiang University（南京理工大学; 南京大学; 浙江大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出动量估计器与小批量技术，实现随机多级组合优化的最优样本复杂度，无需强平均光滑假设，并通过实验验证有效性。
AI 中文摘要
本文研究了随机多级优化问题，其中目标函数是多个光滑非凸函数的嵌套组合。我们假设只能获得每一级梯度和函数值的随机估计。因此，由于嵌套结构，获得整体梯度的准确估计具有挑战性。为了解决这一问题，我们采用基于动量的估计器配合小批量来跟踪每一级的函数值，随后利用这些值构建动量梯度估计器。我们建立了寻找ε-驻点的最优样本复杂度$\nmathcal{O}(\nepsilon^{-4})$，避免了先前文献中常依赖的更强平均光滑性假设。此外，通过采用归一化技术，我们在无需问题相关常数来设置超参数的情况下达到了相同的速率。为了在没有小批量的情况下实现最优速率，我们进一步开发了一种无批量方法，该方法结合了一阶近似和用于函数值估计的裁剪技术。最后，我们通过风险规避投资组合优化和分层倾斜经验风险最小化的实验验证了所提出方法的有效性。
英文摘要
This paper investigates stochastic multi-level optimization where the objective is a nested composition of several smooth non-convex functions. We assume that only stochastic estimates of the gradient and function values for each level are accessible. Consequently, obtaining an accurate estimate of the overall gradient is challenging due to the nested structure. To address this, we employ a momentum-based estimator with mini-batches to track the function values of each level, which are subsequently used to construct momentum gradient estimators. We establish an optimal sample complexity of $\mathcal{O}(ε^{-4})$ for finding an $ε$-stationary point, avoiding the stronger average smoothness assumption commonly relied upon in prior literature. Furthermore, by employing a normalization technique, we attain the same rate without requiring problem-dependent constants to set hyperparameters. To achieve the optimal rate without mini-batches, we further develop a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation. Finally, we validate the effectiveness of our proposed methods through experiments on risk-averse portfolio optimization and hierarchical tilted empirical risk minimization.
