arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1312.2139math.OCcs.ITmath.ITstat.ML
收藏

零级凸优化的最优速率:两个函数的幂估计

Optimal rates for zero-order convex optimization: the power of two function evaluations

John C. Duchi, Michael I. Jordan, Martin J. Wainwright, Andre Wibisono

摘要:

我们考虑的随机和非随机凸优化问题,只使用函数值,而不是梯度的导数免费算法。专注于非渐近界的收敛速度,我们表明,如果对函数值是可用的,算法$d$维优化,使用梯度估计的基础上随机扰动遭受的因素,最多$\sqrt{d}$在收敛速度超过传统的随机梯度法。我们建立这样的结果,光滑和非光滑的情况下,锐化以前的分析,建议一个更坏的尺寸依赖,并将我们的结果的情况下,多个($m \ge 2$)的评价。我们补充我们的算法开发与信息理论的下界上的极小极大收敛速度,这样的问题,建立我们的可实现的结果的锐度恒定(有时是对数)的因素。

英文摘要:

We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients. Focusing on non-asymptotic bounds on convergence rates, we show that if pairs of function values are available, algorithms for $d$-dimensional optimization that use gradient estimates based on random perturbations suffer a factor of at most $\sqrt{d}$ in convergence rate over traditional stochastic gradient methods. We establish such results for both smooth and non-smooth cases, sharpening previous analyses that suggested a worse dimension dependence, and extend our results to the case of multiple ($m \ge 2$) evaluations. We complement our algorithmic development with information-theoretic lower bounds on the minimax convergence rate of such problems, establishing the sharpness of our achievable results up to constant (sometimes logarithmic) factors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑