arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1406.1305math.OCcs.LG
收藏

强凸集上Frank-Wolfe方法的快速度

Faster Rates for the Frank-Wolfe Method over Strongly-Convex Sets

Technion - Israel Institute of Technology(以色列理工学院);Princeton University(普林斯顿大学)

Dan Garber, Elad Hazan

摘要:

Frank-Wolfe方法(a.k.a.近年来,在大规模优化和机器学习的背景下,用于平滑优化的条件梯度算法(conditional gradient algorithm)重新引起了人们的兴趣。该方法的一个关键优点是它避免了投影-许多应用中的计算瓶颈-用线性最优化步骤代替它。尽管有这个优点,FW方法的已知收敛速度对于大多数感兴趣的设置落后于标准的一阶方法。针对各种凸优化的设置导出更快的基于线性最优化的算法是一条活跃的研究路线。本文考虑强凸集上最优化问题的特殊情况,证明了vanila FW方法以$\frac{1}{t^2}$的速度收敛。与一般情况相比,这给出了收敛速度的二次改进,其中收敛是$\frac{1}{t}$阶的,并且已知是紧的。我们证明了由$\ell_p$范数、Schatten范数和群范数诱导的各种球一方面是强凸的,另一方面,在这些集合上的线性最优化是直接的,并且有一个封闭解.我们进一步展示了如何从我们的分析中很容易遵循FW方法的几个以前的快速率结果。

英文摘要:

The Frank-Wolfe method (a.k.a. conditional gradient algorithm) for smooth optimization has regained much interest in recent years in the context of large scale optimization and machine learning. A key advantage of the method is that it avoids projections - the computational bottleneck in many applications - replacing it by a linear optimization step. Despite this advantage, the known convergence rates of the FW method fall behind standard first order methods for most settings of interest. It is an active line of research to derive faster linear optimization-based algorithms for various settings of convex optimization. In this paper we consider the special case of optimization over strongly convex sets, for which we prove that the vanila FW method converges at a rate of $\frac{1}{t^2}$. This gives a quadratic improvement in convergence rate compared to the general case, in which convergence is of the order $\frac{1}{t}$, and known to be tight. We show that various balls induced by $\ell_p$ norms, Schatten norms and group norms are strongly convex on one hand and on the other hand, linear optimization over these sets is straightforward and admits a closed-form solution. We further show how several previous fast-rate results for the FW method follow easily from our analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑