arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1711.07564math.OCstat.CO
收藏

随机函数组合优化的无偏模拟

Unbiased Simulation for Optimizing Stochastic Function Compositions

Jose Blanchet, Donald Goldfarb, Garud Iyengar, Fengpei Li, Chaoxu Zhou

摘要:

本文提出了一种求解随机函数复合凸优化问题的无偏梯度模拟算法。我们表明,从算法产生的无偏梯度具有有限的方差和有限的期望计算成本。然后,我们结合无偏梯度模拟与两个方差减少算法(即SVRG和SCSG),并表明所提出的优化算法的基础上无偏梯度模拟具有令人满意的收敛性能。特别地,在SVRG的情况下,具有模拟梯度的算法可以被证明在期望中线性收敛到最优值,并且在强凸性下几乎必然收敛到最优值。最后,在数值实验中,我们将算法应用于随机函数组合优化的两个重要实例:最大化Cox偏似然模型和训练条件随机场。

英文摘要:

In this paper, we introduce an unbiased gradient simulation algorithms for solving convex optimization problem with stochastic function compositions. We show that the unbiased gradient generated from the algorithm has finite variance and finite expected computation cost. We then combined the unbiased gradient simulation with two variance reduced algorithms (namely SVRG and SCSG) and showed that the proposed optimization algorithms based on unbiased gradient simulations exhibit satisfactory convergence properties. Specifically, in the SVRG case, the algorithm with simulated gradient can be shown to converge linearly to optima in expectation and almost surely under strong convexity. Finally, for the numerical experiment,we applied the algorithms to two important cases of stochastic function compositions optimization: maximizing the Cox's partial likelihood model and training conditional random fields.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑