arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.10652stat.MLcs.LGmath.OC
收藏

机器学习中的蒙特卡罗梯度估计

Monte Carlo Gradient Estimation in Machine Learning

DeepMind;University College London(伦敦大学学院)

Shakir Mohamed, Mihaela Rosca, Michael Figurnov, Andriy Mnih

摘要:

本文是一个广泛的和可访问的调查的方法,我们在我们的处置蒙特卡罗梯度估计在机器学习和整个统计科学:计算梯度的期望函数的参数定义的分布是集成的问题;的问题的敏感性分析。在机器学习研究中,这种梯度问题是监督学习、无监督学习和强化学习中许多学习问题的核心。我们通常会尝试以允许蒙特卡罗估计的形式重写这种梯度,从而使它们能够轻松有效地使用和分析。我们探讨三种策略-路径,得分函数,和测量值梯度估计-探索他们的历史发展,推导,和基本假设。我们描述他们在其他领域的使用,显示它们是如何相关的,可以结合起来,并扩大其可能的概括。无论蒙特卡罗梯度估计已经推导和部署在过去,重要的进展随之而来。对这一问题的更深刻和更广泛的理解将导致进一步的进展,我们希望支持的正是这些进展。

英文摘要:

This paper is a broad and accessible survey of the methods we have at our disposal for Monte Carlo gradient estimation in machine learning and across the statistical sciences: the problem of computing the gradient of an expectation of a function with respect to parameters defining the distribution that is integrated; the problem of sensitivity analysis. In machine learning research, this gradient problem lies at the core of many learning problems, in supervised, unsupervised and reinforcement learning. We will generally seek to rewrite such gradients in a form that allows for Monte Carlo estimation, allowing them to be easily and efficiently used and analysed. We explore three strategies--the pathwise, score function, and measure-valued gradient estimators--exploring their historical development, derivation, and underlying assumptions. We describe their use in other fields, show how they are related and can be combined, and expand on their possible generalisations. Wherever Monte Carlo gradient estimators have been derived and deployed in the past, important advances have followed. A deeper and more widely-held understanding of this problem will lead to further advances, and it is these advances that we wish to support.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑