Generative neural samplers offer a complementary approach to Monte Carlo methods for problems in statistical physics and quantum field theory. This work tests the ability of generative neural samplers to estimate observables for real-world low-dimensional spin systems. It maps out how autoregressive models can sample configurations of a quantum Heisenberg chain via a classical approximation based on the Suzuki-Trotter transformation. We present results for energy, specific heat and susceptibility for the isotropic XXX and the anisotropic XY chain that are in good agreement with Monte Carlo results within the same approximation scheme.
Multilevel Generative Samplers for Investigating Critical Phenomena
BIFOLD；Technische Universität Berlin（技术大学柏林）；Università degli Studi di Torino（都灵大学）；INFN Torino（托斯卡纳国家研究所）；University of Bonn（波恩大学）；Helmholtz Institute for Radiation and Nuclear Physics (HISKP)（辐射和核物理研究所）；Deutsches Elektronen-Synchrotron (DESY)（德国电子同步辐射光源）；RIKEN Center for AIP（理化学研究所AIP中心）
Northeastern University（东北大学）；Massachusetts Institute of Technology（麻省理工学院）
We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frames structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by minimizing the inclusive KL divergence with respect to the conditional posterior. To appropriately account for the size of the input data, we develop a new parameterization in terms of neural sufficient statistics. Experiments show that APG samplers can train highly structured deep g…
Oregon State University（俄勒冈州立大学）；Horizon Robotics（Horizon Robotics（地平线机器人））
Recently, particle-based variational inference (ParVI) methods have gained interest because they can avoid arbitrary parametric assumptions that are common in variational inference. However, many ParVI approaches do not allow arbitrary sampling from the posterior, and the few that do allow such sampling suffer from suboptimality. This work proposes a new method for learning to approximately sample from the posterior distribution. We construct a neural sampler that is trained with the functional gradient of the KL-divergence between the empirical sampling distribution and the target distributio…
Population synthesis is concerned with the generation of synthetic yet realistic representations of populations. It is a fundamental problem in the modeling of transport where the synthetic populations of micro-agents represent a key input to most agent-based models. In this paper, a new methodological framework for how to 'grow' pools of micro-agents is presented. The model framework adopts a deep generative modeling approach from machine learning based on a Variational Autoencoder (VAE). Compared to the previous population synthesis approaches, including Iterative Proportional Fitting (IPF),…
We develop nested variational inference (NVI), a family of methods that learn proposals for nested importance samplers by minimizing an forward or reverse KL divergence at each level of nesting. NVI is applicable to many commonly-used importance sampling strategies and provides a mechanism for learning intermediate densities, which can serve as heuristics to guide the sampler. Our experiments apply NVI to (a) sample from a multimodal distribution using a learned annealing path (b) learn heuristics that approximate the likelihood of future observations in a hidden Markov model and (c) to perfor…
Technische Universität Berlin（柏林工业大学）；Berlin Big Data Center（柏林大数据中心）；RIKEN Center for AIP（理化学研究所人工智能中心）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Berliner Zentrum für Maschinelles Lernen（柏林机器学习中心）；Korea University（高丽大学）；Max-Planck-Institut für Informatik（马克斯·普朗克信息学研究所）
We propose a general framework for the estimation of observables with generative neural samplers focusing on modern deep generative neural networks that provide an exact sampling probability. In this framework, we present asymptotically unbiased estimators for generic observables, including those that explicitly depend on the partition function such as free energy or entropy, and derive corresponding variance estimators. We demonstrate their practical applicability by numerical experiments for the 2d Ising model which highlight the superiority over existing methods. Our approach greatly enhanc…
Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics
ELLIS Unit Linz, LIT AI Lab, Johannes Kepler University Linz, Austria（林茨ELLIS单元、LIT AI实验室、林茨约瑟夫·冯·海德里希大学，奥地利）；Department of Electrical & Computer Engineering, Georgia Institute of Technology（电气与计算机工程系，佐治亚理工学院）；NXAI Lab & NXAI GmbH, Linz, Austria（NXAI实验室及NXAI GmbH，林茨，奥地利）
We develop a framework for approximating collapsed Gibbs sampling in generative latent variable cluster models. Collapsed Gibbs is a popular MCMC method, which integrates out variables in the posterior to improve mixing. Unfortunately for many complex models, integrating out these variables is either analytically or computationally intractable. We efficiently approximate the necessary collapsed Gibbs integrals by borrowing ideas from expectation propagation. We present two case studies where exact collapsed Gibbs sampling is intractable: mixtures of Student-t's and time series clustering. Our …
