Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach
Department of Mathematics, Princeton University（普林斯顿大学数学系）；Department of Mathematics, Duke University（杜克大学数学系）；Department of Physics, and Department of Chemistry, Duke University（杜克大学物理系和杜克大学化学系）
We propose a new method to solve eigenvalue problems for linear and semilinear second order differential operators in high dimensions based on deep neural networks. The eigenvalue problem is reformulated as a fixed point problem of the semigroup flow induced by the operator, whose solution can be represented by Feynman-Kac formula in terms of forward-backward stochastic differential equations. The method shares a similar spirit with diffusion Monte Carlo but augments a direct approximation to the eigenfunction through neural-network ansatz. The criterion of fixed point provides a natural loss function to search for parameters via optimization. Our approach is able to provide accurate eigenvalue and eigenfunction approximations in several numerical examples, including Fokker-Planck operator and the linear and nonlinear Schrödinger operators in high dimensions.
University of Johannesburg（约翰内斯堡大学）；University of Witwatersrand（威特沃特斯兰德大学）
Markov Chain Monte Carlo inference of target posterior distributions in machine learning is predominately conducted via Hamiltonian Monte Carlo and its variants. This is due to Hamiltonian Monte Carlo based samplers ability to suppress random-walk behaviour. As with other Markov Chain Monte Carlo methods, Hamiltonian Monte Carlo produces auto-correlated samples which results in high variance in the estimators, and low effective sample size rates in the generated samples. Adding antithetic sampling to Hamiltonian Monte Carlo has been previously shown to produce higher effective sample rates com…
In this short note we provide an unbiased multilevel Monte Carlo estimator of the log marginal likelihood and discuss its application to variational Bayes.
I consider the problem of integrating a function $f$ over the $d$-dimensional unit cube. I describe a multilevel Monte Carlo method that estimates the integral with variance at most $ε^{2}$ in $O(d+\ln(d)d_{t}ε^{-2})$ time, for $ε>0$, where $d_{t}$ is the truncation dimension of $f$. In contrast, the standard Monte Carlo method typically achieves such variance in $O(dε^{-2})$ time. A lower bound of order $d+d_{t}ε^{-2}$ is described for a class of multilevel Monte Carlo methods.
Sequential Monte Carlo (SMC) samplers form an attractive alternative to MCMC for Bayesian computation. However, their performance depends strongly on the Markov kernels used to rejuvenate particles. We discuss how to calibrate automatically (using the current particles) Hamiltonian Monte Carlo kernels within SMC. To do so, we build upon the adaptive SMC approach of Fearnhead and Taylor (2013), and we also suggest alternative methods. We illustrate the advantages of using HMC kernels within an SMC sampler via an extensive numerical study.
From its inception in the 1950s to the modern frontiers of applied statistics, Markov chain Monte Carlo has been one of the most ubiquitous and successful methods in statistical computing. In that time its development has been fueled by increasingly difficult problems and novel techniques from physics. In this article I will review the history of Markov chain Monte Carlo from its inception with the Metropolis method to today's state-of-the-art in Hamiltonian Monte Carlo. Along the way I will focus on the evolving interplay between the statistical and physical perspectives of the method.
The paper proposes a new Monte-Carlo simulator combining the advantages of Sequential Monte Carlo simulators and Hamiltonian Monte Carlo simulators. The result is a method that is robust to multimodality and complex shapes to use for inference in presence of difficult likelihoods or target functions. Several examples are provided.
A discussion on the possibility of reducing the variance of quasi-Monte Carlo estimators in applications. Further details are provided in the accompanying paper "Variance Reduction for Quasi-Monte Carlo".
Three sampling methods are compared for efficiency on a number of test problems of various complexity for which analytic quadratures are available. The methods compared are Monte Carlo with pseudo-random numbers, Latin Hypercube Sampling, and Quasi Monte Carlo with sampling based on Sobol sequences. Generally results show superior performance of the Quasi Monte Carlo approach based on Sobol sequences in line with theoretical predictions. Latin Hypercube Sampling can be more efficient than both Monte Carlo method and Quasi Monte Carlo method but the latter inequality holds for a reduced set of …
By leveraging the natural geometry of a smooth probabilistic system, Hamiltonian Monte Carlo yields computationally efficient Markov Chain Monte Carlo estimation. At least provided that the algorithm is sufficiently well-tuned. In this paper I show how the geometric foundations of Hamiltonian Monte Carlo implicitly identify the optimal choice of these parameters, especially the integration time. I then consider the practical consequences of these principles in both existing algorithms and a new implementation called \emph{Exhaustive Hamiltonian Monte Carlo} before demonstrating the utility of …
With its systematic exploration of probability distributions, Hamiltonian Monte Carlo is a potent Markov Chain Monte Carlo technique; it is an approach, however, ultimately contingent on the choice of a suitable Hamiltonian function. By examining both the symplectic geometry underlying Hamiltonian dynamics and the requirements of Markov Chain Monte Carlo, we construct the general form of admissible Hamiltonians and propose a particular choice with potential application in Bayesian inference.
ABC (approximate Bayesian computation) is a general approach for dealing with models with an intractable likelihood. In this work, we derive ABC algorithms based on QMC (quasi- Monte Carlo) sequences. We show that the resulting ABC estimates have a lower variance than their Monte Carlo counter-parts. We also develop QMC variants of sequential ABC algorithms, which progressively adapt the proposal distribution and the acceptance threshold. We illustrate our QMC approach through several examples taken from the ABC literature. Keywords: Approximate Bayesian computation, Likelihood-free inference,…
Irreversible and rejection-free Monte Carlo methods, recently developed in Physics under the name Event-Chain and known in Statistics as Piecewise Deterministic Monte Carlo (PDMC), have proven to produce clear acceleration over standard Monte Carlo methods, thanks to the reduction of their random-walk behavior. However, while applying such schemes to standard statistical models, one generally needs to introduce an additional randomization for sake of correctness. We propose here a new class of Event-Chain Monte Carlo methods that reduces this extra-randomization to a bare minimum. We compare t…
This file is the Sweave documentation for the examples provided in Flegal, J. M. and Jones, G. L. (2010), "Implementing Markov chain Monte Carlo: Estimating with confidence", in Handbook of Markov Chain Monte Carlo, edited by Brooks, S., Gelman, A., Jones, G., and Meng, X. published by Chapman & Hall/CRC Press.
We describe an "embarrassingly parallel" method for Bayesian phylogenetic inference, annealed Sequential Monte Carlo, based on recent advances in the Sequential Monte Carlo literature such as adaptive determination of annealing parameters. The algorithm provides an approximate posterior distribution over trees and evolutionary parameters as well as an unbiased estimator for the marginal likelihood. This unbiasedness property can be used for the purpose of testing the correctness of posterior simulation software. We evaluate the performance of phylogenetic annealed Sequential Monte Carlo by rev…
Statisticians often use Monte Carlo methods to approximate probability distributions, primarily with Markov chain Monte Carlo and importance sampling. Sequential Monte Carlo samplers are a class of algorithms that combine both techniques to approximate distributions of interest and their normalizing constants. These samplers originate from particle filtering for state space models and have become general and scalable sampling techniques. This article describes sequential Monte Carlo samplers and their possible implementations, arguing that they remain under-used in statistics, despite their ab…
Leveraging the coherent exploration of Hamiltonian flow, Hamiltonian Monte Carlo produces computationally efficient Monte Carlo estimators, even with respect to complex and high-dimensional target distributions. When confronted with data-intensive applications, however, the algorithm may be too expensive to implement, leaving us to consider the utility of approximations such as data subsampling. In this paper I demonstrate how data subsampling fundamentally compromises the efficient exploration of Hamiltonian flow and hence the scalable performance of Hamiltonian Monte Carlo itself.
We show that Lasso and Bayesian Lasso are very close when the sparsity is large and the noise is small. Then we propose to solve Bayesian Lasso using multivalued stochastic differential equation. We obtain three discretizations algorithms, and propose a method for calculating the cost of Monte-Carlo (MC), multilevel Monte Carlo (MLMC) and MCMC algorithms.
In this paper, we develop Bayesian Hamiltonian Monte Carlo methods for inference in asymmetric GARCH models under different distributions for the error term. We implemented Zero-variance and Hamiltonian Monte Carlo schemes for parameter estimation to try and reduce the standard errors of the estimates thus obtaing more efficient results at the price of a small extra computational cost.
We present bounds for the finite sample error of sequential Monte Carlo samplers on static spaces. Our approach explicitly relates the performance of the algorithm to properties of the chosen sequence of distributions and mixing properties of the associated Markov kernels. This allows us to give the first finite sample comparison to other Monte Carlo schemes. We obtain bounds for the complexity of sequential Monte Carlo approximations for a variety of target distributions including finite spaces, product measures, and log-concave distributions including Bayesian logistic regression. The bounds…
This is the solution manual to the odd-numbered exercises in our book "Introducing Monte Carlo Methods with R", published by Springer Verlag on December 10, 2009, and made freely available to everyone.
Estimating the density of a continuous random variable X has been studied extensively in statistics, in the setting where n independent observations of X are given a priori and one wishes to estimate the density from that. Popular methods include histograms and kernel density estimators. In this review paper, we are interested instead in the situation where the observations are generated by Monte Carlo simulation from a model. Then, one can take advantage of variance reduction methods such as stratification, conditional Monte Carlo, and randomized quasi-Monte Carlo (RQMC), and obtain a more ac…
Markov chain Monte Carlo (MCMC) is a simulation method commonly used for estimating expectations with respect to a given distribution. We consider estimating the covariance matrix of the asymptotic multivariate normal distribution of a vector of sample means. Geyer (1992) developed a Monte Carlo error estimation method for estimating a univariate mean. We propose a novel multivariate version of Geyer's method that provides an asymptotically valid estimator for the covariance matrix and results in stable Monte Carlo estimates. The finite sample properties of the proposed method are investigated…
Markov Chain Monte Carlo (MCMC) is an invaluable means of inference with complicated models, and Hamiltonian Monte Carlo, in particular Riemannian Manifold Hamiltonian Monte Carlo (RMHMC), has demonstrated impressive success in many challenging problems. Current RMHMC implementations, however, rely on a Riemannian metric that limits their application to analytically-convenient models. In this paper I propose a new metric for RMHMC without these limitations and verify its success on a distribution that emulates many hierarchical and latent models.
Monte Carlo methods are now an essential part of the statistician's toolbox, to the point of being more familiar to graduate students than the measure theoretic notions upon which they are based! We recall in this note some of the advances made in the design of Monte Carlo techniques towards their use in Statistics, referring to Robert and Casella (2004,2010) for an in-depth coverage.
The partition function of a factor graph can sometimes be accurately estimated by Monte Carlo methods. In this paper, such methods are extended to factor graphs with negative and complex factors.
We present a Cross-Entropy based population Monte Carlo algorithm. This methods stands apart from previous work in that we are not optimizing a mixture distribution. Instead, we leverage deterministic mixture weights and optimize the distributions individually through a reinterpretation of the typical derivation of the cross-entropy method. Demonstrations on numerical examples show that the algorithm can outperform existing resampling population Monte Carlo methods, especially for higher-dimensional problems.
In this paper we address the problem of Monte Carlo approximation of posterior probability distributions in stochastic kinetic models (SKMs). SKMs are multivariate Markov jump processes that model the interactions among species in biochemical systems according to a set of uncertain parameters. Markov chain Monte Carlo (MCMC) methods have been typically preferred for this Bayesian inference problem. Specifically, the particle MCMC (pMCMC) method has been recently shown to be an effective, while computationally demanding, method applicable to this problem. Within the pMCMC framework, importance …
We consider quantile estimation using Markov chain Monte Carlo and establish conditions under which the sampling distribution of the Monte Carlo error is approximately Normal. Further, we investigate techniques to estimate the associated asymptotic variance, which enables construction of an asymptotically valid interval estimator. Finally, we explore the finite sample properties of these methods through examples and provide some recommendations to practitioners.
Pricing options is an important problem in financial engineering. In many scenarios of practical interest, financial option prices associated to an underlying asset reduces to computing an expectation w.r.t.~a diffusion process. In general, these expectations cannot be calculated analytically, and one way to approximate these quantities is via the Monte Carlo method; Monte Carlo methods have been used to price options since at least the 1970's. It has been seen in Del Moral, P. \& Shevchenko, P.V. (2014) `Valuation of barrier options using Sequential Monte Carlo' and Jasra, A. \& Del Moral, P.…
We model online recommendation systems using the hidden Markov multi-state restless multi-armed bandit problem. To solve this we present Monte Carlo rollout policy. We illustrate numerically that Monte Carlo rollout policy performs better than myopic policy for arbitrary transition dynamics with no specific structure. But, when some structure is imposed on the transition dynamics, myopic policy performs better than Monte Carlo rollout policy.
U. of Southern California（南加州大学）；U. de Montréal（蒙特利尔大学）；Google Research（谷歌研究院）
Reinforcement learning constantly deals with hard integrals, for example when computing expectations in policy evaluation and policy iteration. These integrals are rarely analytically solvable and typically estimated with the Monte Carlo method, which induces high variance in policy values and gradients. In this work, we propose to replace Monte Carlo samples with low-discrepancy point sets. We combine policy gradient methods with Randomized Quasi-Monte Carlo, yielding variance-reduced formulations of policy gradient and actor-critic algorithms. These formulations are effective for policy eval…
