arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.11154stat.MLcs.LGstat.ME
收藏

并行MCMC无尴尬故障

Parallel MCMC Without Embarrassing Failures

University College London(伦敦大学学院);Aalto University(阿尔托大学);Getulio Vargas Foundation(热图利奥·瓦加斯基金会);University of Manchester(曼彻斯特大学);University of Helsinki(赫尔辛基大学)

Daniel Augusto de Souza, Diego Mesquita, Samuel Kaski, Luigi Acerbi

摘要:

令人尴尬的并行马尔可夫链蒙特卡罗(MCMC)利用并行计算,通过使用两步方法将贝叶斯推断扩展到大型数据集。首先,MCMC在数据分区上定义的(子)后验上并行运行。然后,服务器组合本地结果。虽然有效,这个框架是非常敏感的次后验抽样的质量。常见的采样问题,如缺失模式或低密度区域的错误表示,在组合阶段被放大-而不是被纠正-导致灾难性的故障。在这项工作中,我们提出了一种新的组合策略来缓解这个问题。我们的策略,并行主动推理(PAI),利用高斯过程(GP)代理建模和主动学习。在将GP拟合到次后验后,PAI(i)在GP代理之间共享信息以覆盖缺失的模式;以及(ii)使用主动采样来单独地改进次后验近似。我们验证PAI在具有挑战性的基准,包括重尾和多模态后验和现实世界中的应用计算神经科学。实证结果表明,PAI成功地在以前的方法灾难性地失败,与一个小的通信开销。

英文摘要:

Embarrassingly parallel Markov Chain Monte Carlo (MCMC) exploits parallel computing to scale Bayesian inference to large datasets by using a two-step approach. First, MCMC is run in parallel on (sub)posteriors defined on data partitions. Then, a server combines local results. While efficient, this framework is very sensitive to the quality of subposterior sampling. Common sampling problems such as missing modes or misrepresentation of low-density regions are amplified -- instead of being corrected -- in the combination phase, leading to catastrophic failures. In this work, we propose a novel combination strategy to mitigate this issue. Our strategy, Parallel Active Inference (PAI), leverages Gaussian Process (GP) surrogate modeling and active learning. After fitting GPs to subposteriors, PAI (i) shares information between GP surrogates to cover missing modes; and (ii) uses active sampling to individually refine subposterior approximations. We validate PAI in challenging benchmarks, including heavy-tailed and multi-modal posteriors and a real-world application to computational neuroscience. Empirical results show that PAI succeeds where previous methods catastrophically fail, with a small communication overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑