We present a specific algorithm that generally satisfies the balance condition without imposing the detailed balance in the Markov chain Monte Carlo. In our algorithm, the average rejection rate is minimized, and even reduced to zero in many relevant cases. The absence of the detailed balance also introduces a net stochastic flow in a configuration space, which further boosts up the convergence. We demonstrate that the autocorrelation time of the Potts model becomes more than 6 times shorter than that by the conventional Metropolis algorithm. Based on the same concept, a bounce-free worm algorithm for generic quantum spin models is formulated as well.
Among the Markov chains breaking detailed-balance that have been proposed in the field of Monte-Carlo sampling in order to accelerate the convergence towards the steady state with respect to the detailed-balance dynamics, the idea of 'Lifting' consists in duplicating the configuration space into two copies $σ=\pm$ and in imposing directed flows in each copy in order to explore the configuration space more efficiently. The skew-detailed-balance Lifted-Markov-chain introduced by K. S. Turitsyn, M. Chertkov and M. Vucelja [Physica D Nonlinear Phenomena 240 , 410 (2011)] is revisited for the Curie…
We define a class of Markovian parallel dynamics for spin systems on arbitrary graphs with nearest neighbor interaction described by a Hamiltonian function $H(σ)$. These dynamics turn out to be reversible and their stationary measure is explicitly determined. Convergence to equilibrium and relation of the stationary measure to the usual Gibbs measure are discussed when the dynamics is defined on $\mathbb{Z}^2$. Further it is shown how these dynamics can be used to define natively parallel algorithms to face problems in the context of combinatorial optimization.
Adaptive Monte Carlo methods can be viewed as implementations of Markov chains with infinite memory. We derive a general condition for the convergence of a Monte Carlo method whose history dependence is contained within the simulated density distribution. In convergent cases, our result implies that the balance condition need only be satisfied asymptotically. As an example, we show that the adaptive integration method converges.
Time-reversal symmetry of microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the detailed balance. On the other hand, cyclic Markov processes that do not admit equilibrium distributions with detailed balance, are often used to model open systems driven out of equilibrium by external agents. I show that for a Markov model without detailed balance, an extended Markov model that explicitly includes the degrees of freedom for the driving agent can be constructed, such that the original cyclic Markov model for the driven system can be recovered as an appro…
Detailed balance is an overly strict condition to ensure a valid Monte Carlo simulation. We show that, under fairly general assumptions, a Monte Carlo simulation need satisfy only the weaker balance condition. Not only does our proof show that sequential updating schemes are correct, but also it establishes the correctness of a whole class of new methods that simply leave the Boltzmann distribution invariant.
We consider the optimization of Markovian dynamics to pursue the fastest convergence to the stationary state. The brachistochrone method is applied to the continuous-time master equation for finite-size systems. The principle of least action leads to a brachistochrone equation for the transition-rate matrix. Three-state systems are explicitly analyzed, and we find that the solution violates the detailed balance condition. The properties of the solution are studied in detail to observe the optimality of the solution. We also discuss the counterdiabatic driving for the Markovian dynamics. The tr…
We propose a new Markov chain Monte Carlo method in which trial configurations are generated by evolving a state, sampled from a prior distribution, using a Markov transition matrix. We present two prototypical algorithms and derive their corresponding acceptance rules. We first identify the important factors controlling the quality of the sampling. We then apply the method to the problem of sampling polymer configurations with fixed endpoints. Applications of the proposed method range from the design of new generative models to the improvement of the portability of specific Monte Carlo algori…
A definition of metastable states applicable to arbitrary finite state Markov processes satisfying detailed balance is discussed. In particular, we identify a crucial condition that distinguishes genuine metastable states from other types of slowly decaying modes and which leads to properties similar to those postulated in the restricted ensemble approach \cite{pen71}. The intuitive physical meaning of this condition is simply that the total equilibrium probability of finding the system in the metastable state is negligible. As a concrete application of our formalism we present preliminary res…
Sampling occupies an important position in theories of various scientific fields, and Markov chain Monte Carlo (MCMC) provides the most common technique of sampling. In the progress of MCMC, a huge number of studies have aimed the acceleration of convergence to the target distribution. Hamiltonian Monte Carlo (HMC) is such a variant of MCMC. In the recent development of MCMC, another approach based on the violation of the detailed balance condition has attracted much attention. Historically, these two approaches have been proposed independently, and their relationship has not been clearly unde…
This is an introductory article about Markov Chain Monte Carlo (MCMC) simulation for pedestrians. Actual simulation codes are provided, and necessary practical details, which are skipped in most textbooks, are shown. The second half is written for hep-th and hep-lat audience. It explains specific methods needed for simulations with dynamical fermions, especially supersymmetric Yang-Mills. The examples include QCD and matrix integral, in addition to SYM.
A review of Bayesian restoration of digital images based on Monte Carlo techniques is presented. The topics covered include Likelihood, Prior and Posterior distributions, Poisson, Binay symmetric channel, and Gaussian channel models of Likelihood distribution,Ising and Potts spin models of Prior distribution, restoration of an image through Posterior maximization, statistical estimation of a true image from Posterior ensembles, Markov Chain Monte Carlo methods and cluster algorithms.
The Markov chain Monte Carlo method is a versatile tool in statistical physics to evaluate multi-dimensional integrals numerically. For the method to work effectively, we must consider the following key issues: the choice of ensemble, the selection of candidate states, the optimization of transition kernel, algorithm for choosing a configuration according to the transition probabilities. We show that the unconventional approaches based on the geometric allocation of probabilities or weights can improve the dynamics and scaling of the Monte Carlo simulation in several aspects. Particularly, the…
We introduce an efficient nonreversible Markov chain Monte Carlo algorithm to generate self-avoiding walks with a variable endpoint. In two dimensions, the new algorithm slightly outperforms the two-move nonreversible Berretti-Sokal algorithm introduced by H.~Hu, X.~Chen, and Y.~Deng in \cite{old}, while for three-dimensional walks, it is 3--5 times faster. The new algorithm introduces nonreversible Markov chains that obey global balance and allows for three types of elementary moves on the existing self-avoiding walk: shorten, extend or alter conformation without changing the walk's length.
We propose a minimal generalization of the celebrated Markov-Chain Monte Carlo algorithm which allows for an arbitrary number of configurations to be visited at every Monte Carlo step. This is advantageous when a parallel computing machine is available, or when many biased configurations can be evaluated at little additional computational cost. As an example of the former case, we report a significant reduction of the thermalization time for the paradigmatic Sherrington-Kirkpatrick spin-glass model. For the latter case, we show that, by leveraging on the exponential number of biased configurat…
Markov chain Monte Carlo algorithms are invaluable tools for exploring stationary properties of physical systems, especially in situations where direct sampling is unfeasible. Common implementations of Monte Carlo algorithms employ reversible Markov chains. Reversible chains obey detailed balance and thus ensure that the system will eventually relax to equilibrium. Detailed balance is not necessary for convergence to equilibrium. We review nonreversible Markov chains, which violate detailed balance, and yet still relax to a given target stationary distribution. In particular cases, nonreversib…
In this article, we present an event-driven algorithm that generalizes the recent hard-sphere event-chain Monte Carlo method without introducing discretizations in time or in space. A factorization of the Metropolis filter and the concept of infinitesimal Monte Carlo moves are used to design a rejection-free Markov-chain Monte Carlo algorithm for particle systems with arbitrary pairwise interactions. The algorithm breaks detailed balance, but satisfies maximal global balance and performs better than the classic, local Metropolis algorithm in large systems. The new algorithm generates a continu…
We propose a method to construct a proposal density for the Metropolis-Hastings algorithm in Markov Chain Monte Carlo (MCMC) simulations of the GARCH model. The proposal density is constructed adaptively by using the data sampled by the MCMC metho d itself. It turns out that autocorrelations between the data generated with our adaptive proposal density are greatly reduced. Thus it is concluded that the adaptive construction method is very efficient and works well for the MCMC simulations of the GARCH model.
We present a history-dependent Monte Carlo scheme for the efficient calculation of the free-energy of quantum systems, inspired by the Wang-Landau sampling and metadynamics method. When embedded in a path integral formulation, it is of general applicability to a large variety of Hamiltonians. In the two-dimensional quantum Ising model, chosen here for illustration, the accuracy of free energy, critical temperature, and specific heat is demonstrated as a function of simulation time, and successfully compared with the best available approaches, particularly the Wang-Landau method over two differ…
This article is a tutorial on Markov chain Monte Carlo simulations and their statistical analysis. The theoretical concepts are illustrated through many numerical assignments from the author's book on the subject. Computer code (in Fortran) is available for all subjects covered and can be downloaded from the web.
Perimeter Institute for Theoretical Physics（圆周理论物理研究所）；University of Cambridge（剑桥大学）；University of Waterloo（滑铁卢大学）；Massachusetts Institute of Technology（麻省理工学院）
A Markov chain update scheme using a machine-learned flow-based generative model is proposed for Monte Carlo sampling in lattice field theories. The generative model may be optimized (trained) to produce samples from a distribution approximating the desired Boltzmann distribution determined by the lattice action of the theory being studied. Training the model systematically improves autocorrelation times in the Markov chain, even in regions of parameter space where standard Markov chain Monte Carlo algorithms exhibit critical slowing down in producing decorrelated updates. Moreover, the model …
The computer revolution has been driven by a sustained increase of computational speed of approximately one order of magnitude (a factor of ten) every five years since about 1950. In natural sciences this has led to a continuous increase of the importance of computer simulations. Major enabling techniques are Markov Chain Monte Carlo (MCMC) and Molecular Dynamics (MD) simulations. This article deals with the MCMC approach. First basic simulation techniques, as well as methods for their statistical analysis are reviewed. Afterwards the focus is on generalized ensembles and biased updating, two …
