arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.04096math.OC
收藏

采用开环步长的Frank-Wolfe算法的加速仿射不变收敛速率

Accelerated Affine-Invariant Convergence Rates of the Frank-Wolfe Algorithm with Open-Loop Step-Sizes

Elias Wirth, Javier Pena, Sebastian Pokutta

AI总结:

针对现有开环步长Frank-Wolfe算法加速收敛结果依赖仿射变换的问题,本文结合仿射不变性与开环步长研究脉络,将其非仿射不变收敛速率推广为仿射不变结果,填补了相关研究空白。

摘要:

近期研究表明,采用开环步长的Frank-Wolfe算法(FW)的收敛速率快于经典的$\u2112(t^{-1})$速率。具体而言,当多胞体上强凸函数的极小值点位于可行域某个面的相对内部时,采用开环步长$η_t = \frac{\u2113}{t+\u2113}$(其中$\u2113 \u2208 \u2115_{\u2265 2}$)的FW算法可实现$\u2112(t^{-2})$的加速收敛,而更复杂的线搜索或短步长步长仅能达到$Ω(t^{-1-ε})$的速率。鉴于该场景在数据科学问题中的重要性,探索开环FW算法实现加速的适用场景的研究日益增多。然而,尽管FW算法具有众所周知的仿射不变性,现有开环FW的加速结果却依赖仿射变换。本文结合仿射不变性(Wirth等人,2023b)与开环步长(Pena,2021)两条最新研究脉络,弥补了文献中的这一空白,将所有已知的开环步长FW算法的非仿射不变收敛速率推广为仿射不变结果。

英文摘要:

Recent papers have shown that the Frank-Wolfe algorithm (FW) with open-loop step-sizes exhibits rates of convergence faster than the iconic $\mathcal{O}(t^{-1})$ rate. In particular, when the minimizer of a strongly convex function over a polytope lies in the relative interior of a feasible region face, the FW with open-loop step-sizes $η_t = \frac{\ell}{t+\ell}$ for $\ell \in \mathbb{N}_{\geq 2}$ has accelerated convergence $\mathcal{O}(t^{-2})$ in contrast to the rate $Ω(t^{-1-ε})$ attainable with more complex line-search or short-step step-sizes. Given the relevance of this scenario in data science problems, research has grown to explore the settings enabling acceleration in open-loop FW. However, despite FW's well-known affine invariance, existing acceleration results for open-loop FW are affine-dependent. This paper remedies this gap in the literature by merging two recent research trajectories: affine invariance (Wirth et al., 2023b) and open-loop step-sizes (Pena, 2021). In particular, we extend all known non-affine-invariant convergence rates for FW with open-loop step-sizes to affine-invariant results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑