arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1811.08413stat.MLcs.LG
收藏

采样可以比优化更快

Sampling Can Be Faster Than Optimization

University of California, Berkeley(加州大学伯克利分校)

Yi-An Ma, Yuansi Chen, Chi Jin, Nicolas Flammarion, Michael I. Jordan

摘要:

优化算法和蒙特卡罗抽样算法为近年来统计机器学习应用的快速增长提供了计算基础。然而,对这两种方法之间的关系的理论理解有限,对相对优势和劣势的理解有限。此外,现有的结果主要是在设置凸函数(优化)和对数凹函数(采样)。在这种情况下,局部属性决定全局属性,优化算法在计算上比采样算法更有效。相反,我们研究了一类非凸目标函数,出现在混合建模和多稳定系统。在这个非凸的设置,我们发现,采样算法的计算复杂度与模型维数成线性比例,而优化算法的规模呈指数。

英文摘要:

Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in recent years. There is, however, limited theoretical understanding of the relationships between these two kinds of methodology, and limited understanding of relative strengths and weaknesses. Moreover, existing results have been obtained primarily in the setting of convex functions (for optimization) and log-concave functions (for sampling). In this setting, where local properties determine global properties, optimization algorithms are unsurprisingly more efficient computationally than sampling algorithms. We instead examine a class of nonconvex objective functions that arise in mixture modeling and multi-stable systems. In this nonconvex setting, we find that the computational complexity of sampling algorithms scales linearly with the model dimension while that of optimization algorithms scales exponentially.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑