A Gibbs Sampling Alternative to Reversible Jump MCMC

Reversible jump MCMC for nonparametric drift estimation for diffusion processes

In the context of nonparametric Bayesian estimation a Markov chain Monte Carlo algorithm is devised and implemented to sample from the posterior distribution of the drift function of a continuously or discretely observed one-dimensional diffusion. The drift is modeled by a scaled linear combination of basis functions with a Gaussian prior on the coefficients. The scaling parameter is equipped with a partially conjugate prior. The number of basis function in the drift is equipped with a prior distribution as well. For continuous data, a reversible jump Markov chain algorithm enables the explora…

Asynchronous Gibbs Sampling

Gibbs sampling is a Markov Chain Monte Carlo (MCMC) method often used in Bayesian learning. MCMC methods can be difficult to deploy on parallel and distributed systems due to their inherently sequential nature. We study asynchronous Gibbs sampling, which achieves parallelism by simply ignoring sequential requirements. This method has been shown to produce good empirical results for some hierarchical models, and is popular in the topic modeling community, but was also shown to diverge for other targets. We introduce a theoretical framework for analyzing asynchronous Gibbs sampling and other ext…

在SMC²中自适应切换粒子边缘Metropolis-Hastings核与粒子Gibbs核

Adaptively switching between a particle marginal Metropolis-Hastings and a particle Gibbs kernel in SMC$^2$

针对难以计算似然的状态空间模型后验采样问题，提出在SMC²迭代中自适应切换PMMH和PG粒子MCMC核的方法，显著提升算法效率并开源代码。

gemlib.mcmc: 用于Metropolis-within-Gibbs采样方案的可组合内核

gemlib.mcmc: composable kernels for Metropolis-within-Gibbs sampling schemes

本文提出gemlib.mcmc框架，通过可组合内核抽象简化MCMC实现，利用范畴论中的writer monads实现内核组合，降低开发难度，使研究人员能高效应用先进算法。

Particle Gibbs Sampling for Bayesian Phylogenetic inference

The combinatorial sequential Monte Carlo (CSMC) has been demonstrated to be an efficient complementary method to the standard Markov chain Monte Carlo (MCMC) for Bayesian phylogenetic tree inference using biological sequences. It is appealing to combine the CSMC and MCMC in the framework of the particle Gibbs (PG) sampler to jointly estimate the phylogenetic trees and evolutionary parameters. However, the Markov chain of the particle Gibbs may mix poorly if the underlying SMC suffers from the path degeneracy issue. Some remedies, including the particle Gibbs with ancestor sampling and the inte…

Population-Based Reversible Jump Markov Chain Monte Carlo

In this paper we present an extension of population-based Markov chain Monte Carlo (MCMC) to the trans-dimensional case. One of the main challenges in MCMC-based inference is that of simulating from high and trans-dimensional target measures. In such cases, MCMC methods may not adequately traverse the support of the target; the simulation results will be unreliable. We develop population methods to deal with such problems, and give a result proving the uniform ergodicity of these population algorithms, under mild assumptions. This result is used to demonstrate the superiority, in terms of conv…

Efficient MCMC for Gibbs Random Fields using pre-computation

Bayesian inference of Gibbs random fields (GRFs) is often referred to as a doubly intractable problem, since the likelihood function is intractable. The exploration of the posterior distribution of such models is typically carried out with a sophisticated Markov chain Monte Carlo (MCMC) method, the exchange algorithm (Murray et al., 2006), which requires simulations from the likelihood function at each iteration. The purpose of this paper is to consider an approach to dramatically reduce this computational overhead. To this end we introduce a novel class of algorithms which use realizations of…

Importance sampling correction versus standard averages of reversible MCMCs in terms of the asymptotic variance

We establish an ordering criterion for the asymptotic variances of two consistent Markov chain Monte Carlo (MCMC) estimators: an importance sampling (IS) estimator, based on an approximate reversible chain and subsequent IS weighting, and a standard MCMC estimator, based on an exact reversible chain. Essentially, we relax the criterion of the Peskun type covariance ordering by considering two different invariant probabilities, and obtain, in place of a strict ordering of asymptotic variances, a bound of the asymptotic variance of IS by that of the direct MCMC. Simple examples show that IS can …

Adaptive Gibbs samplers and related MCMC methods

We consider various versions of adaptive Gibbs and Metropolis-within-Gibbs samplers, which update their selection probabilities (and perhaps also their proposal distributions) on the fly during a run by learning as they go in an attempt to optimize the algorithm. We present a cautionary example of how even a simple-seeming adaptive Gibbs sampler may fail to converge. We then present various positive results guaranteeing convergence of adaptive Gibbs samplers under certain conditions.

Supplement to "Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs"

This supplementary material includes three parts: some preliminary results, four examples, an experiment, three new algorithms, and all proofs of the results in the paper "Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs".

Reversible Genetically Modified Mode Jumping MCMC

In this paper, we introduce a reversible version of a genetically modified mode jumping Markov chain Monte Carlo algorithm (GMJMCMC) for inference on posterior model probabilities in complex model spaces, where the number of explanatory variables is prohibitively large for classical Markov Chain Monte Carlo methods. Unlike the earlier proposed GMJMCMC algorithm, the introduced algorithm is a proper MCMC and its limiting distribution corresponds to the posterior marginal model probabilities in the explored model space under reasonable regularity conditions.

具有梯度漂移和常数扩散的随机微分方程的精确吉布斯采样

Exact Gibbs sampling for stochastic differential equations with gradient drift and constant diffusion

本文提出了一种适用于广泛SDE类别的精确吉布斯采样方法，无需离散化误差，通过变换扩展到更广泛的多元SDEs，并在合成和真实数据上展示了优于粒子MCMC的性能。

Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better

I show how any reversible Markov chain on a finite state space that is irreducible, and hence suitable for estimating expectations with respect to its invariant distribution, can be used to construct a non-reversible Markov chain on a related state space that can also be used to estimate these expectations, with asymptotic variance at least as small as that using the reversible chain (typically smaller). The non-reversible chain achieves this improvement by avoiding (to the extent possible) transitions that backtrack to the state from which the chain just came. The proof that this modification…

Particle MCMC with Poisson Resampling: Parallelization and Continuous Time Models

We introduce a new version of particle filter in which the number of "children" of a particle at a given time has a Poisson distribution. As a result, the number of particles is random and varies with time. An advantage of this scheme is that descendants of different particles can evolve independently. It makes easy to parallelize computations. Moreover, particle filter with Poisson resampling is readily adapted to the case when a hidden process is a continuous time, piecewise deterministic semi-Markov process. We show that the basic techniques of particle MCMC, namely particle independent Met…

On Bayesian inference for the M/G/1 queue with efficient MCMC sampling

We introduce an efficient MCMC sampling scheme to perform Bayesian inference in the M/G/1 queueing model given only observations of interdeparture times. Our MCMC scheme uses a combination of Gibbs sampling and simple Metropolis updates together with three novel "shift" and "scale" updates. We show that our novel updates improve the speed of sampling considerably, by factors of about 60 to about 180 on a variety of simulated data sets.

Control Variates for Reversible MCMC Samplers

A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible MCMC samplers. We propose the use of a specific class of functions as control variates, and we introduce a new, consistent estimator for the values of the coefficients of the optimal linear combination of these functions. The form and proposed construction of the control variates is derived from our solution of the Poisson equation associated with a specific MCMC scenario. The new estimator, which can be applied to the same MCMC sample, is…

通过深度展开学习替换分裂吉布斯扩散后验采样中的MCMC

Learning to Replace MCMC in Split-Gibbs Diffusion Posterior Sampling via Deep Unfolding

提出用深度展开网络替换分裂吉布斯采样中的MCMC步骤，通过ODE扩散实现高斯去噪更新，在非线性相位恢复中实现更低成本的扩散后验采样。

Sequential Gibbs Sampling Algorithm for Cognitive Diagnosis Models with Many Attributes

Cognitive diagnosis models (CDMs) are useful statistical tools to provide rich information relevant for intervention and learning. As a popular approach to estimate and make inference of CDMs, the Markov chain Monte Carlo (MCMC) algorithm is widely used in practice. However, when the number of attributes, $K$, is large, the existing MCMC algorithm may become time-consuming, due to the fact that $O(2^K)$ calculations are usually needed in the process of MCMC sampling to get the conditional distribution for each attribute profile. To overcome this computational issue, motivated by Culpepper and …

Non-reversible Markov chain Monte Carlo for sampling of districting maps

Evaluating the degree of partisan districting (Gerrymandering) in a statistical framework typically requires an ensemble of districting plans which are drawn from a prescribed probability distribution that adheres to a realistic and non-partisan criteria. In this article we introduce novel non-reversible Markov chain Monte-Carlo (MCMC) methods for the sampling of such districting plans which have improved mixing properties in comparison to previously used (reversible) MCMC algorithms. In doing so we extend the current framework for construction of non-reversible Markov chains on discrete sampl…

Reversible Jump MCMC Simulated Annealing for Neural Networks

We propose a novel reversible jump Markov chain Monte Carlo (MCMC) simulated annealing algorithm to optimize radial basis function (RBF) networks. This algorithm enables us to maximize the joint posterior distribution of the network parameters and the number of basis functions. It performs a global search in the joint space of the parameters and number of parameters, thereby surmounting the problem of local minima. We also show that by calibrating a Bayesian model, we can obtain the classical AIC, BIC and MDL model selection criteria within a penalized likelihood framework. Finally, we show th…

使用 R 理解 Gibbs 采样

Digesting Gibbs Sampling Using R

本文面向希望在研究中使用 MCMC（尤其是 Gibbs 采样）的用户，提供基于 R 的实践框架，并说明读者所需的微积分与统计学预备知识。

DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs

Performing reliable Bayesian inference on a big data scale is becoming a keystone in the modern era of machine learning. A workhorse class of methods to achieve this task are Markov chain Monte Carlo (MCMC) algorithms and their design to handle distributed datasets has been the subject of many works. However, existing methods are not completely either reliable or computationally efficient. In this paper, we propose to fill this gap in the case where the dataset is partitioned and stored on computing nodes within a cluster under a master/slaves architecture. We derive a user-friendly centralise…

使用可逆跳跃MCMC求解马尔可夫决策过程的新推理策略

New inference strategies for solving Markov Decision Processes using reversible jump MCMC

本文提出基于可逆跳跃MCMC的改进推理策略，通过新目标分布和打破参数-轨迹相关性，实现高维空间中的最优策略估计。

Model comparison for Gibbs random fields using noisy reversible jump Markov chain Monte Carlo

The reversible jump Markov chain Monte Carlo (RJMCMC) method offers an across-model simulation approach for Bayesian estimation and model comparison, by exploring the sampling space that consists of several models of possibly varying dimensions. A naive implementation of RJMCMC to models like Gibbs random fields suffers from computational difficulties: the posterior distribution for each model is termed doubly-intractable since computation of the likelihood function is rarely available. Consequently, it is simply impossible to simulate a transition of the Markov chain in the presence of likeli…

量化 MCMC 回火算法中非可逆性带来的加速

Quantifying the Speed-Up from Non-Reversibility in MCMC Tempering Algorithms

本文研究了非可逆更新赋予“动量”后对模拟与并行回火 MCMC 算法效率的提升，证明其最优缩放版本比传统可逆版本仅有约 42% 的适度加速。

用于潜在动态系统中可并行化推断的辅助 MCMC 和粒子 Gibbs 采样器

Auxiliary MCMC and particle Gibbs samplers for parallelisable inference in latent dynamical systems

针对高维潜在动态模型采样难题，提出引入人工观测作为辅助变量的 MCMC 核，开发出高效精确的卡尔曼采样器与增强 Particle Gibbs 算法，支持 GPU 时间维度并行化，显著提升了高维推断性能。

Convergence Rates of Two-Component MCMC Samplers

Component-wise MCMC algorithms, including Gibbs and conditional Metropolis-Hastings samplers, are commonly used for sampling from multivariate probability distributions. A long-standing question regarding Gibbs algorithms is whether a deterministic-scan (systematic-scan) sampler converges faster than its random-scan counterpart. We answer this question when the samplers involve two components by establishing an exact quantitative relationship between the $L^2$ convergence rates of the two samplers. The relationship shows that the deterministic-scan sampler converges faster. We also establish q…

Approximate spectral gaps for Markov chains mixing times in high dimensions

This paper introduces a concept of approximate spectral gap to analyze the mixing time of Markov Chain Monte Carlo (MCMC) algorithms for which the usual spectral gap is degenerate or almost degenerate. We use the idea to analyze a class of MCMC algorithms to sample from mixtures of densities. As an application we study the mixing time of a Gibbs sampler for variable selection in linear regression models. Under some regularity conditions on the signal and the design matrix of the regression problem, we show that for well-chosen initial distributions the mixing time of the Gibbs sampler is polyn…

GPU-accelerated Gibbs sampling: a case study of the Horseshoe Probit model

Gibbs sampling is a widely used Markov chain Monte Carlo (MCMC) method for numerically approximating integrals of interest in Bayesian statistics and other mathematical sciences. Many implementations of MCMC methods do not extend easily to parallel computing environments, as their inherently sequential nature incurs a large synchronization cost. In the case study illustrated by this paper, we show how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit, for a large class of exchangeable models that admit latent variable representations. Our approach takes a syste…

Complexity Results for MCMC derived from Quantitative Bounds

This paper considers how to obtain MCMC quantitative convergence bounds which can be translated into tight complexity bounds in high-dimensional {settings}. We propose a modified drift-and-minorization approach, which establishes generalized drift conditions defined in subsets of the state space. The subsets are called the "large sets", and are chosen to rule out some "bad" states which have poor drift property when the dimension of the state space gets large. Using the "large sets" together with a "fitted family of drift functions", a quantitative bound can be obtained which can be translated…

Prune Sampling: a MCMC inference technique for discrete and deterministic Bayesian networks

We introduce and characterise the performance of the Markov chain Monte Carlo (MCMC) inference method Prune Sampling for discrete and deterministic Bayesian networks (BNs). We developed a procedure to obtain the performance of a MCMC sampling method in the limit of infinite simulation time, extrapolated from relatively short simulations. This approach was used to conduct a study to compare the accuracy, rate of convergence and the time consumption of Prune Sampling with two conventional MCMC sampling methods: Gibbs- and Metropolis sampling. We show that Markov chains created by Prune Sampling …

用于离散结构与不可逆系统发育MCMC的Zig-zag采样

Zig-zag sampling for discrete structures and non-reversible phylogenetic MCMC

本文提出一种适用于离散-连续混合状态空间的Zig-zag采样方法，无需假设离散变量结构，在基于Kingman溯祖过程的遗传学实例中相较Metropolis-Hastings算法实现高达数个数量级的效率提升，并避免了哈密顿蒙特卡洛的边界穿越复杂性。

Mini-batch Metropolis-Hastings MCMC with Reversible SGLD Proposal

Traditional MCMC algorithms are computationally intensive and do not scale well to large data. In particular, the Metropolis-Hastings (MH) algorithm requires passing over the entire dataset to evaluate the likelihood ratio in each iteration. We propose a general framework for performing MH-MCMC using mini-batches of the whole dataset and show that this gives rise to approximately a tempered stationary distribution. We prove that the algorithm preserves the modes of the original target distribution and derive an error bound on the approximation with mild assumptions on the likelihood. To furthe…

Posterior computation with the Gibbs zig-zag sampler

An intriguing new class of piecewise deterministic Markov processes (PDMPs) has recently been proposed as an alternative to Markov chain Monte Carlo (MCMC). In order to facilitate the application to a larger class of problems, we propose a new class of PDMPs termed Gibbs zig-zag samplers, which allow parameters to be updated in blocks with a zig-zag sampler applied to certain parameters and traditional MCMC-style updates to others. We demonstrate the flexibility of this framework on posterior sampling for logistic models with shrinkage priors for high-dimensional regression and random effects …

Stability of Conditional Sequential Monte Carlo

The particle Gibbs (PG) sampler is a Markov Chain Monte Carlo (MCMC) algorithm, which uses an interacting particle system to perform the Gibbs steps. Each Gibbs step consists of simulating a particle system conditioned on one particle path. It relies on a conditional Sequential Monte Carlo (cSMC) method to create the particle system. We propose a novel interpretation of the cSMC algorithm as a perturbed Sequential Monte Carlo (SMC) method and apply telescopic decompositions developed for the analysis of SMC algorithms \cite{delmoral2004} to derive a bound for the distance between the expected …

A novel Empirical Bayes with Reversible Jump Markov Chain in User-Movie Recommendation system

In this article we select the unknown dimension of the feature by re- versible jump MCMC inside a simulated annealing in bayesian set up of collaborative filter. We implement the same in MovieLens small dataset. We also tune the hyper parameter by using a modified empirical bayes. It can also be used to guess an initial choice for hyper-parameters in grid search procedure even for the datasets where MCMC oscillates around the true value or takes long time to converge.

Parameter elimination in particle Gibbs sampling

Bayesian inference in state-space models is challenging due to high-dimensional state trajectories. A viable approach is particle Markov chain Monte Carlo, combining MCMC and sequential Monte Carlo to form "exact approximations" to otherwise intractable MCMC methods. The performance of the approximation is limited to that of the exact method. We focus on particle Gibbs and particle Gibbs with ancestor sampling, improving their performance beyond that of the underlying Gibbs sampler (which they approximate) by marginalizing out one or more parameters. This is possible when the parameter prior i…

Accelerated Sampling on Discrete Spaces with Non-Reversible Markov Processes

We consider the task of MCMC sampling from a distribution defined on a discrete space. Building on recent insights provided in [Zan19], we devise a class of efficient continuous-time, non-reversible algorithms which make active use of the structure of the underlying space. Particular emphasis is placed on how symmetries and other group-theoretic notions can be used to improve exploration of the space. We test our algorithms on a range of examples from statistics, computational physics, machine learning, and cryptography, which show improvement on alternative algorithms. We provide practical re…

Faster MCMC for Gaussian Latent Position Network Models

Latent position network models are a versatile tool in network science; applications include clustering entities, controlling for causal confounders, and defining priors over unobserved graphs. Estimating each node's latent position is typically framed as a Bayesian inference problem, with Metropolis within Gibbs being the most popular tool for approximating the posterior distribution. However, it is well-known that Metropolis within Gibbs is inefficient for large networks; the acceptance ratios are expensive to compute, and the resultant posterior draws are highly correlated. In this article,…

Fast Doubly-Adaptive MCMC to Estimate the Gibbs Partition Function with Weak Mixing Time Bounds

We present a novel method for reducing the computational complexity of rigorously estimating the partition functions (normalizing constants) of Gibbs (Boltzmann) distributions, which arise ubiquitously in probabilistic graphical models. A major obstacle to practical applications of Gibbs distributions is the need to estimate their partition functions. The state of the art in addressing this problem is multi-stage algorithms, which consist of a cooling schedule, and a mean estimator in each step of the schedule. While the cooling schedule in these algorithms is adaptive, the mean estimation com…

Improving the Asymptotic Performance of Markov Chain Monte-Carlo by Inserting Vortices

We present a new way of converting a reversible finite Markov chain into a non-reversible one, with a theoretical guarantee that the asymptotic variance of the MCMC estimator based on the non-reversible chain is reduced. The method is applicable to any reversible chain whose states are not connected through a tree, and can be interpreted graphically as inserting vortices into the state transition graph. Our result confirms that non-reversible chains are fundamentally better than reversible ones in terms of asymptotic performance, and suggests interesting directions for further improving MCMC.

Markov Interacting Importance Samplers

We introduce a new Markov chain Monte Carlo (MCMC) sampler called the Markov Interacting Importance Sampler (MIIS). The MIIS sampler uses conditional importance sampling (IS) approximations to jointly sample the current state of the Markov Chain and estimate conditional expectations, possibly by incorporating a full range of variance reduction techniques. We compute Rao-Blackwellized estimates based on the conditional expectations to construct control variates for estimating expectations under the target distribution. The control variates are particularly efficient when there are substantial c…

使用公共随机数模拟估计MCMC收敛速率

Estimating MCMC convergence rates using common random number simulation

本文利用公共随机数模拟为MCMC收敛提供Wasserstein距离上界，并在多个模型中验证其比传统漂移界收敛更快，速率与单次耦合界相当。

Irreversible Langevin MCMC on Lie Groups

It is well-known that irreversible MCMC algorithms converge faster to their stationary distributions than reversible ones. Using the special geometric structure of Lie groups $\mathcal G$ and dissipation fields compatible with the symplectic structure, we construct an irreversible HMC-like MCMC algorithm on $\mathcal G$, where we first update the momentum by solving an OU process on the corresponding Lie algebra $\mathfrak g$, and then approximate the Hamiltonian system on $\mathcal G \times \mathfrak g$ with a reversible symplectic integrator followed by a Metropolis-Hastings correction step.…

修改Gibbs采样以避免自转移

Modifying Gibbs sampling to avoid self transitions

本文回顾并提出了多种修改Gibbs采样的可逆与不可逆方法以减少自转移并降低渐近方差，在多个模型模拟中验证了性能，发现顺序更新的不可逆性有益但单个不可逆更新无一致优势。

Techniques for proving Asynchronous Convergence results for Markov Chain Monte Carlo methods

Markov Chain Monte Carlo (MCMC) methods such as Gibbs sampling are finding widespread use in applied statistics and machine learning. These often lead to difficult computational problems, which are increasingly being solved on parallel and distributed systems such as compute clusters. Recent work has proposed running iterative algorithms such as gradient descent and MCMC in parallel asynchronously for increased performance, with good empirical results in certain problems. Unfortunately, for MCMC this parallelization technique requires new convergence theory, as it has been explicitly demonstra…

Fast MCMC sampling for Markov jump processes and extensions

Markov jump processes (or continuous-time Markov chains) are a simple and important class of continuous-time dynamical systems. In this paper, we tackle the problem of simulating from the posterior distribution over paths in these models, given partial and noisy observations. Our approach is an auxiliary variable Gibbs sampler, and is based on the idea of uniformization. This sets up a Markov chain over paths by alternately sampling a finite set of virtual jump times given the current path and then sampling a new path given the set of extant and virtual jump times using a standard hidden Marko…

A Polynomial Time MCMC Method for Sampling from Continuous DPPs

We study the Gibbs sampling algorithm for continuous determinantal point processes. We show that, given a warm start, the Gibbs sampler generates a random sample from a continuous $k$-DPP defined on a $d$-dimensional domain by only taking $\text{poly}(k)$ number of steps. As an application, we design an algorithm to generate random samples from $k$-DPPs defined by a spherical Gaussian kernel on a unit sphere in $d$-dimensions, $\mathbb{S}^{d-1}$ in time polynomial in $k,d$.

在序列长度上并行化MCMC

Parallelizing MCMC Across the Sequence Length

本文提出在序列长度上并行化MCMC的方法，通过并行牛顿方法加速采样，提升效率。

Particle-Gibbs Sampling For Bayesian Feature Allocation Models

Bayesian feature allocation models are a popular tool for modelling data with a combinatorial latent structure. Exact inference in these models is generally intractable and so practitioners typically apply Markov Chain Monte Carlo (MCMC) methods for posterior inference. The most widely used MCMC strategies rely on an element wise Gibbs update of the feature allocation matrix. These element wise updates can be inefficient as features are typically strongly correlated. To overcome this problem we have developed a Gibbs sampler that can update an entire row of the feature allocation matrix in a s…

On particle Gibbs sampling

The particle Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm to sample from the full posterior distribution of a state-space model. It does so by executing Gibbs sampling steps on an extended target distribution defined on the space of the auxiliary variables generated by an interacting particle system. This paper makes the following contributions to the theoretical study of this algorithm. Firstly, we present a coupling construction between two particle Gibbs updates from different starting points and we show that the coupling probability may be made arbitrarily close to one by i…

利用活动子空间改进 MCMC

Improved MCMC with active subspaces

本文针对活动子空间 MCMC 在线性假设失效时的局限，提出粒子边缘 MH，并给出无需估计边缘似然的 MWG 与 MWPG 方法以提升计算效率。

Note on the computation of the Metropolis-Hastings ratio for Birth-or-Death moves in trans-dimensional MCMC algorithms for signal decomposition problems

Reversible jump MCMC (RJ-MCMC) sampling techniques, which allow to jointly tackle model selection and parameter estimation problems in a coherent Bayesian framework, have become increasingly popular in the signal processing literature since the seminal paper of Andrieu and Doucet (IEEE Trans. Signal Process., 47(10), 1999). Crucial to the implementation of any RJ-MCMC sampler is the computation of the so-called Metropolis-Hastings-Green (MHG) ratio, which determines the acceptance probability for the proposed moves. It turns out that the expression of the MHG ratio that was given in the paper …

用于贝叶斯神经网络的近似分块 Gibbs 采样

Approximate blocked Gibbs sampling for Bayesian neural networks

本文提出近似分块 Gibbs 采样以提升前馈神经网络小批量 MCMC 的可行性，通过参数分组与深层降方差缓解深度增加导致的接受率消失，从而支持更长链运行并改善分类准确率与不确定性量化。