随机约束组合优化的无投影多级算法

中文摘要 AI 辅助

本文研究随机约束多级组合优化的无投影算法。在此背景下，目标函数是多个光滑函数的嵌套组合，决策集是闭凸集。由于投影到约束集可能计算代价高昂，我们开发了依赖线性最小化预言机的无投影方法。对于非凸目标，我们提出了方差缩减的无投影算法，并在Frank-Wolfe间隙和梯度映射准则下建立了复杂度保证。我们还开发了基于动量的方法，在较弱的平滑性假设下实现收敛保证。此外，通过采用分阶段设计，我们推导出一个无参数变体，该变体对Frank-Wolfe间隙保持相同的复杂度。这种设计可进一步用于开发凸函数和强凸函数的算法，其速率与单级无投影对应算法相匹配。最后，我们考虑有限和问题，并推导非凸、凸和强凸目标的复杂度。跨多个任务的数值实验证明了所提出方法的有效性。

英文摘要

This paper studies projection-free algorithms for stochastic constrained multi-level compositional optimization. In this context, the objective function is a nested composition of several smooth functions, and the decision set is closed and convex. Since projection onto the constraint set can be computationally expensive, we develop projection-free methods that rely on linear minimization oracles. For non-convex objectives, we propose variance-reduced projection-free algorithms and establish complexity guarantees under both the Frank-Wolfe gap and the gradient mapping criteria. We also develop momentum-based methods that achieve convergence guarantees under weaker smoothness assumptions. Additionally, by using a stage-wise design, we derive a parameter-free variant that preserves the same complexities for the Frank-Wolfe gap. Such a design can be further used to develop algorithms for convex and strongly convex functions whose rates match those of single-level projection-free counterparts. Finally, we consider finite-sum problems and derive complexities for non-convex, convex, and strongly convex objectives. Numerical experiments across multiple tasks demonstrate the effectiveness of the proposed methods.

发表机构

• Nanjing University of Science and Technology（南京理工大学）
• Nanjing University（南京大学）
• Zhejiang University（浙江大学）

机构由 AI 辅助整理，请以论文原文为准。