arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.00610stat.MLcs.LG
收藏

用于采样多尺度分布的延迟拒绝 Hamiltonian Monte Carlo

Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions

Flatiron Institute(熨斗研究所)

Chirag Modi, Alex Barnett, Bob Carpenter

AI总结:

针对多尺度分布中 HMC 步长受高曲率区域限制的问题,提出几何递减步长的延迟拒绝 HMC,并扩展重试概率机制,最高带来五倍采样效率提升。

摘要:

Hamiltonian Monte Carlo(HMC)在采样具有广泛长度尺度范围的分布时,效率可能会受损,因为高曲率区域为保持稳定性所需的小步长在其他区域效率低下。为解决这一问题,我们提出一种延迟拒绝变体:如果初始 HMC 轨迹被拒绝,我们会提出一个或多个后续提案,每个提案使用比上一个提案按几何级数更小的步长。我们扩展了标准延迟拒绝框架,允许重试概率依赖于接受前一提案的概率。我们在若干采样任务中测试了该方案,包括 Neal's funnel 等多尺度模型分布以及统计应用。以每次梯度评估的有效样本量衡量,延迟拒绝相比经过最优调参的 HMC 可实现最高五倍的性能提升。即使对于较简单的分布,延迟拒绝也增强了对步长设定不当的鲁棒性。在此过程中,我们还对 HMC 的细致平衡进行了通俗易懂且严谨的综述。

英文摘要:

The efficiency of Hamiltonian Monte Carlo (HMC) can suffer when sampling a distribution with a wide range of length scales, because the small step sizes needed for stability in high-curvature regions are inefficient elsewhere. To address this we present a delayed rejection variant: if an initial HMC trajectory is rejected, we make one or more subsequent proposals each using a step size geometrically smaller than the last. We extend the standard delayed rejection framework by allowing the probability of a retry to depend on the probability of accepting the previous proposal. We test the scheme in several sampling tasks, including multiscale model distributions such as Neal's funnel, and statistical applications. Delayed rejection enables up to five-fold performance gains over optimally-tuned HMC, as measured by effective sample size per gradient evaluation. Even for simpler distributions, delayed rejection provides increased robustness to step size misspecification. Along the way, we provide an accessible but rigorous review of detailed balance for HMC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑