arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1307.7192cs.LGmath.OC
收藏

MixedGrad:一个O(1/T)收敛速度的随机光滑优化算法

MixedGrad: An O(1/T) Convergence Rate Algorithm for Stochastic Smooth Optimization

Michigan State University(密歇根州立大学)

Mehrdad Mahdavi, Rong Jin

摘要:

光滑函数随机优化的最优收敛速度是$O(1/\sqrt{T})$,这与Lipschitz连续凸函数随机优化的最优收敛速度相同。这与使用全梯度优化平滑函数形成对比,全梯度产生$O(1/T^2)$的收敛速率。在这项工作中,我们考虑一个新的设置优化光滑函数,称为{ \bf混合优化},它允许访问随机预言和一个完整的梯度预言。我们的目标是显着提高光滑函数的随机优化的收敛速度有一个额外的少量的访问完整的梯度预言。我们表明,与$O(\ln T)$调用全梯度预言和$O(T)$调用随机预言,建议的混合优化算法是能够实现的优化误差$O(1/T)$。

英文摘要:

It is well known that the optimal convergence rate for stochastic optimization of smooth functions is $O(1/\sqrt{T})$, which is same as stochastic optimization of Lipschitz continuous convex functions. This is in contrast to optimizing smooth functions using full gradients, which yields a convergence rate of $O(1/T^2)$. In this work, we consider a new setup for optimizing smooth functions, termed as {\bf Mixed Optimization}, which allows to access both a stochastic oracle and a full gradient oracle. Our goal is to significantly improve the convergence rate of stochastic optimization of smooth functions by having an additional small number of accesses to the full gradient oracle. We show that, with an $O(\ln T)$ calls to the full gradient oracle and an $O(T)$ calls to the stochastic oracle, the proposed mixed optimization algorithm is able to achieve an optimization error of $O(1/T)$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑