Current objective functions used for training neural MCMC proposal distributions implicitly rely on architectural restrictions to yield sensible optimization results, which hampers the development of highly expressive neural MCMC proposal architectures. In this work, we introduce and demonstrate a semi-empirical procedure for determining approximate objective functions suitable for optimizing arbitrarily parameterized proposal distributions in MCMC methods. Our proposed Ab Initio objective functions consist of the weighted combination of functions following constraints on their global optima and transformation invariances that we argue should be upheld by general measures of MCMC efficiency for use in proposal optimization. Our experimental results demonstrate that Ab Initio objective functions maintain favorable performance and preferable optimization behavior compared to existing objective functions for neural MCMC optimization. We find that Ab Initio objective functions are sufficiently robust to enable the confident optimization of neural proposal distributions parameterized by deep generative networks extending beyond the regimes of traditional MCMC schemes
A Neural Network MCMC sampler that maximizes Proposal Entropy
Redwood Center for theoretical Neuroscience(雷德伍德理论神经科学中心);Department of Physics(物理系);University of California Berkeley(加州大学伯克利分校);Berkeley AI Research(伯克利人工智能研究中心);Helen Wills Neuroscience Institute(海伦·威尔斯神经科学研究所)
Scaling-up Split-Merge MCMC with Locality Sensitive Sampling (LSS)
Rice University(莱斯大学)
分裂-合并MCMC(Monte Carlo Markov Chain)是MCMC的一种基本和流行的变体,用于解决MCMC状态由未知数量的组件组成的问题。众所周知,用于分裂-合并MCMC的最新方法不能很好地扩展。快速混合的策略需要智能和信息丰富的建议,以减少拒绝率。然而,所有已知的智能建议都涉及昂贵的操作来建议信息转换。因此,每次迭代的成本对于大规模数据集是令人望而却步的。进一步已知的是,无信息但计算有效的建议,如随机分裂-合并,导致收敛非常慢。混合时间和每次更新成本之间的权衡似乎很难解决。在本文中,我们展示了一个甜蜜点。我们利用加权MinHash的一些独特属性,这是一种流行的LSH,设计了一类新的分裂合并建议,这比随机采样的信息量更大,但同时计算效率更高。总的来说,我们得到了一个优越的收敛和每次更新成本之间的权衡。作为一个直接的结果,我们的建议是大约6倍的速度比最先进的采样方法在两个大型的真实数据集KDDCUP和PubMed与数百万个实体和数千个集群。
IsingFormer: Augmenting Parallel Tempering With Learned Proposals
Department of Electrical and Computer Engineering, University of California, Santa Barbara(加州大学圣芭芭拉分校电气与计算机工程系);Electrical Engineering Department, King Fahd University of Petroleum & Minerals (KFUPM)(国王法赫德石油与矿物大学(KFUPM)电子工程系)
Bayesian neural networks via MCMC: a Python-based tutorial
School of Mathematics and Statistics, UNSW Sydney(新南威尔士大学悉尼分校数学与统计学院);Pingala Institute of Artificial Intelligence(平加拉人工智能研究院);University of Sydney(悉尼大学)
Metropolis-Hastings view on variational inference and adversarial training
National Research University Higher School of Economics(国家研究型高等经济学院);Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院);Samsung AI Center Moscow(三星莫斯科人工智能中心);HSE(高等经济学院);Skoltech(斯科尔科沃科学技术研究院)
Variational Autoencoder for Generating Broader-Spectrum prior Proposals in Markov chain Monte Carlo Methods
Department One(部门一);Department Two(部门二);National Laboratory for Scientific Computing(国家科学计算实验室);Department of Mathematical Sciences, The University of Texas at Dallas(数学科学系,德克萨斯大学达拉斯分校)
MetFlow: A New Efficient Method for Bridging the Gap between Markov Chain Monte Carlo and Variational Inference
CMAP, Ecole Polytechnique, Universite Paris-Saclay(巴黎萨克雷大学综合理工学院CMAP实验室);CDISE, Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究所CDISE实验室);Ecole Normale Supérieure Paris-Saclay(巴黎萨克雷高等师范学院)
Incorporating the ChEES Criterion into Sequential Monte Carlo Samplers
Dept. of EEE, University of Liverpool, United Kingdom(电子工程系,利物浦大学,英国);Intelligent Sensor-Actuator-Systems Laboratory (ISAS), Karlsruhe Institute of Technology, Germany(智能传感器-执行器系统实验室(ISAS),卡尔斯鲁厄理工学院,德国)