Normalizing flows for lattice gauge theory in arbitrary space-time dimension
Massachusetts Institute of Technology（麻省理工学院）；New York University（纽约大学）；DeepMind（深度思维）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Bern（伯尔尼大学）
Ryan Abbott, Michael S. Albergo, Aleksandar Botev, Denis Boyda, Kyle Cranmer, Daniel C. Hackett, Gurtej Kanwar, Alexander G. D. G. Matthews, Sébastien Racanière… 展开作者
Ryan Abbott, Michael S. Albergo, Aleksandar Botev, Denis Boyda, Kyle Cranmer, Daniel C. Hackett, Gurtej Kanwar, Alexander G. D. G. Matthews, Sébastien Racanière, Ali Razavi, Danilo J. Rezende, Fernando Romero-López, Phiala E. Shanahan, Julian M. Urban
Applications of normalizing flows to the sampling of field configurations in lattice gauge theory have so far been explored almost exclusively in two space-time dimensions. We report new algorithmic developments of gauge-equivariant flow architectures facilitating the generalization to higher-dimensional lattice geometries. Specifically, we discuss masked autoregressive transformations with tractable and unbiased Jacobian determinants, a key ingredient for scalable and asymptotically exact flow-based sampling algorithms. For concreteness, results from a proof-of-principle application to SU(3) lattice gauge theory in four space-time dimensions are reported.
Normalizing flows are constructed from a base distribution with a known density and a diffeomorphism with a tractable Jacobian. The base density of a normalizing flow can be parameterised by a different normalizing flow, thus allowing maps to be found between arbitrary distributions. We demonstrate and explore the utility of this approach and show it is particularly interesting in the case of conditional normalizing flows and for introducing optimal transport constraints on maps that are constructed using normalizing flows.
New York University（纽约大学）；Argonne National Laboratory（阿贡国家实验室）；Massachusetts Institute of Technology（麻省理工学院）；DeepMind（深度心智）
This notebook tutorial demonstrates a method for sampling Boltzmann distributions of lattice field theories using a class of machine learning models known as normalizing flows. The ideas and approaches proposed in arXiv:1904.12072, arXiv:2002.02428, and arXiv:2003.06413 are reviewed and a concrete implementation of the framework is presented. We apply this framework to a lattice scalar field theory and to U(1) gauge theory, explicitly encoding gauge symmetries in the flow-based approach to the latter. This presentation is intended to be interactive and working with the attached Jupyter noteboo…
Normalizing flows are a class of deep generative models that provide a promising route to sample lattice field theories more efficiently than conventional Monte Carlo simulations. In this work we show that the theoretical framework of stochastic normalizing flows, in which neural-network layers are combined with Monte Carlo updates, is the same that underlies out-of-equilibrium simulations based on Jarzynski's equality, which have been recently deployed to compute free-energy differences in lattice gauge theories. We lay out a strategy to optimize the efficiency of this extended class of gener…
Non-Perturbative Trivializing Flows for Lattice Gauge Theories
Institute of Physics, University of Amsterdam（阿姆斯特丹大学物理研究所）；Department of Computing, Imperial College London（伦敦帝国学院计算机系）；Institute for Mathematics, Academia Sinica, Taiwan（台湾“中央研究院”数学研究所）；Korteweg-de Vries Institute for Mathematics, University of Amsterdam（阿姆斯特丹大学柯尔特韦格-德维尔斯数学研究所）
Coupling-based normalizing flows (e.g. RealNVP) are a popular family of normalizing flow architectures that work surprisingly well in practice. This calls for theoretical understanding. Existing work shows that such flows weakly converge to arbitrary data distributions. However, they make no statement about the stricter convergence criterion used in practice, the maximum likelihood loss. For the first time, we make a quantitative statement about this kind of convergence: We prove that all coupling-based normalizing flows perform whitening of the data distribution (i.e. diagonalize the covarian…
Normalizing flows are a powerful technique for obtaining reparameterizable samples from complex multimodal distributions. Unfortunately, current approaches are only available for the most basic geometries and fall short when the underlying space has a nontrivial topology, limiting their applicability for most real-world data. Using fundamental ideas from differential geometry and geometric control theory, we describe how the recently introduced Neural ODEs and continuous normalizing flows can be extended to arbitrary smooth manifolds. We propose a general methodology for parameterizing vector …
Technische Universität Berlin（柏林工业大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；RIKEN Center for AIP（理化学研究所先进智能项目中心）
We propose an algorithm to estimate the path-gradient of both the reverse and forward Kullback-Leibler divergence for an arbitrary manifestly invertible normalizing flow. The resulting path-gradient estimators are straightforward to implement, have lower variance, and lead not only to faster convergence of training but also to better overall approximation results compared to standard total gradient estimators. We also demonstrate that path-gradient training is less susceptible to mode-collapse. In light of our results, we expect that path-gradient estimators will become the new standard method…
University of Waterloo（滑铁卢大学）；Vector Institute（矢量研究所）；University of Edinburgh（爱丁堡大学）
Density deconvolution is the task of estimating a probability density function given only noise-corrupted samples. We can fit a Gaussian mixture model to the underlying density by maximum likelihood if the noise is normally distributed, but would like to exploit the superior density estimation performance of normalizing flows and allow for arbitrary noise distributions. Since both adjustments lead to an intractable likelihood, we resort to amortized variational inference. We demonstrate some problems involved in this approach, however, experiments on real data demonstrate that flows can alread…
Unsupervised anomaly detection is coming into the spotlight these days in various practical domains due to the limited amount of anomaly data. One of the major approaches for it is a normalizing flow which pursues the invertible transformation of a complex distribution as images into an easy distribution as N(0, I). In fact, algorithms based on normalizing flow like FastFlow and CFLOW-AD establish state-of-the-art performance on unsupervised anomaly detection tasks. Nevertheless, we investigate these algorithms convert normal images into not N(0, I) as their destination, but an arbitrary norma…
The sampling of probability distributions specified up to a normalization constant is an important problem in both machine learning and statistical mechanics. While classical stochastic sampling methods such as Markov Chain Monte Carlo (MCMC) or Langevin Dynamics (LD) can suffer from slow mixing times there is a growing interest in using normalizing flows in order to learn the transformation of a simple prior distribution to the given target distribution. Here we propose a generalized and combined approach to sample target densities: Stochastic Normalizing Flows (SNF) -- an arbitrary sequence …
As 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point clouds becomes crucial. Despite the recent success of deep learning models in discriminative tasks of point clouds, generating point clouds remains challenging. This paper proposes a principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes and the…
Emory University（埃默里大学）；University of California, Los Angeles（加州大学洛杉矶分校）
A normalizing flow is an invertible mapping between an arbitrary probability distribution and a standard normal distribution; it can be used for density estimation and statistical inference. Computing the flow follows the change of variables formula and thus requires invertibility of the mapping and an efficient way to compute the determinant of its Jacobian. To satisfy these requirements, normalizing flows typically consist of carefully chosen components. Continuous normalizing flows (CNFs) are mappings obtained by solving a neural ordinary differential equation (ODE). The neural ODE's dynami…
TU Wien（维也纳工业大学）；Institute for Theoretical Physics（理论物理研究所）
The introduction of relevant physical information into neural network architectures has become a widely used and successful strategy for improving their performance. In lattice gauge theories, such information can be identified with gauge symmetries, which are incorporated into the network layers of our recently proposed Lattice Gauge Equivariant Convolutional Neural Networks (L-CNNs). L-CNNs can generalize better to differently sized lattices than traditional neural networks and are by construction equivariant under lattice gauge transformations. In these proceedings, we present our progress …
University of California, Berkeley（加州大学伯克利分校）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
We develop an iterative (greedy) deep learning (DL) algorithm which is able to transform an arbitrary probability distribution function (PDF) into the target PDF. The model is based on iterative Optimal Transport of a series of 1D slices, matching on each slice the marginal PDF to the target. The axes of the orthogonal slices are chosen to maximize the PDF difference using Wasserstein distance at each iteration, which enables the algorithm to scale well to high dimensions. As special cases of this algorithm, we introduce two sliced iterative Normalizing Flow (SINF) models, which map from the d…
Generative models, such as the method of normalizing flows, have been suggested as alternatives to the standard algorithms for generating lattice gauge field configurations. Studies with the method of normalizing flows demonstrate the proof of principle for simple models in two dimensions. However, further studies indicate that the training cost can be, in general, very high for large lattices. The poor scaling traits of current models indicate that moderate-size networks cannot efficiently handle the inherently multi-scale aspects of the problem, especially around critical points. We explore …
Conditional Normalizing Flows for Forward and Backward Joint State and Parameter Estimation
Department of Computer Science and Engineering University of Notre Dame（计算机科学与工程系诺特达姆大学）；Department of Pediatrics Stanford University（儿科系斯坦福大学）；Department of Applied and Computational Mathematics and Statistics University of Notre Dame（应用与计算数学与统计系诺特达姆大学）
Building on the recent trend of new deep generative models known as Normalizing Flows (NF), simulation-based inference (SBI) algorithms can now efficiently accommodate arbitrary complex and high-dimensional data distributions. The development of appropriate validation methods however has fallen behind. Indeed, most of the existing metrics either require access to the true posterior distribution, or fail to provide theoretical guarantees on the consistency of the inferred approximation beyond the one-dimensional setting. This work proposes easy to interpret validation diagnostics for multi-dime…
Amortized Sampling with Transferable Normalizing Flows
University of Oxford（牛津大学）；Université de Montréal（蒙特利尔大学）；Mila - Quebec AI Institute（魁北克AI研究所）；Freie Universität Berlin（柏林自由大学）；Valence Labs（Valence实验室）；AITHYRA；Institut Courtois（科特伊研究所）
To better conform to data geometry, recent deep generative modelling techniques adapt Euclidean constructions to non-Euclidean spaces. In this paper, we study normalizing flows on manifolds. Previous work has developed flow models for specific cases; however, these advancements hand craft layers on a manifold-by-manifold basis, restricting generality and inducing cumbersome design constraints. We overcome these issues by introducing Neural Manifold Ordinary Differential Equations, a manifold generalization of Neural ODEs, which enables the construction of Manifold Continuous Normalizing Flows …
Normalizing flows attempt to model an arbitrary probability distribution through a set of invertible mappings. These transformations are required to achieve a tractable Jacobian determinant that can be used in high-dimensional scenarios. The first normalizing flow designs used coupling layer mappings built upon affine transformations. The significant advantage of such models is their easy-to-compute inverse. Nevertheless, making use of affine transformations may limit the expressiveness of such models. Recently, invertible piecewise polynomial functions as a replacement for affine transformati…
Princeton University（普林斯顿大学）；New York University（纽约大学）；Flatiron Institute（熨斗研究所）
We demonstrate an algorithm for learning a flexible color-magnitude diagram from noisy parallax and photometry measurements using a normalizing flow, a deep neural network capable of learning an arbitrary multi-dimensional probability distribution. We present a catalog of 640M photometric distance posteriors to nearby stars derived from this data-driven model using Gaia DR2 photometry and parallaxes. Dust estimation and dereddening is done iteratively inside the model and without prior distance information, using the Bayestar map. The signal-to-noise (precision) of distance measurements improv…
UT Austin（德克萨斯大学奥斯汀分校）；Princeton University（普林斯顿大学）
We study image inverse problems with a normalizing flow prior. Our formulation views the solution as the maximum a posteriori estimate of the image conditioned on the measurements. This formulation allows us to use noise models with arbitrary dependencies as well as non-linear forward operators. We empirically validate the efficacy of our method on various inverse problems, including compressed sensing with quantized measurements and denoising with highly structured noise patterns. We also present initial theoretical recovery guarantees for solving inverse problems with a flow prior.
The recent introduction of machine learning techniques, especially normalizing flows, for the sampling of lattice gauge theories has shed some hope on improving the sampling efficiency of the traditional HMC algorithm. Naive use of normalizing flows has been shown to lead to bad scaling with the volume. In this talk we propose using local normalizing flows at a scale given by the correlation length. Even if naively these transformations have a small acceptance, when combined with the HMC algorithm lead to algorithms with high acceptance, and also with reduced autocorrelation times compared wit…
We propose a unifying approach that starts from the perturbative construction of trivializing maps by Lüscher and then improves on it by learning. The resulting continuous normalizing flow model can be implemented using common tools of lattice field theory and requires several orders of magnitude fewer parameters than any existing machine learning approach. Specifically, our model can achieve competitive performance with as few as 14 parameters while existing deep-learning models have around 1 million parameters for $SU(3)$ Yang--Mills theory on a $16^2$ lattice. This has obvious consequences …
Machine learning methods based on normalizing flows have been shown to address important challenges, such as critical slowing-down and topological freezing, in the sampling of gauge field configurations in simple lattice field theories. A critical question is whether this success will translate to studies of QCD. This Proceedings presents a status update on advances in this area. In particular, it is illustrated how recently developed algorithmic components may be combined to construct flow-based sampling algorithms for QCD in four dimensions. The prospects and challenges for future use of thi…
Axial anomaly of lattice abelian gauge theory in hyper-cubic regular lattice in arbitrary even dimensions is investigated by applying the method of exterior differential calculus. The topological invariance, gauge invariance and locality of the axial anomaly determine the explicit form of the topological part. The anomaly is obtained up to a multiplicative constant for finite lattice spacing and can be interpreted as the Chern character of the abelian lattice gauge theory.
We study the non-perturbative production of gluon pairs from a constant SU(3) chromo-electric background field via the Schwinger mechanism. We fix the covariant background gauge with an arbitrary gauge parameter α. We determine the transverse momentum distribution of the gluons, as well as the total probability of creating pairs per unit space time volume. We find that the result is independent of the covariant gauge parameter αused to define arbitrary covariant background gauges. We find that our non-perturbative result is both gauge invariant and gauge parameter αindependent.
One-loop off-shell contributions to the three-gluon vertex are calculated, in arbitrary covariant gauge and in arbitrary space-time dimension, including quark-loop contributions (with massless quarks). It is shown how one can get the results for all on-shell limits of interest directly from the general off-shell expression. The corresponding general expressions for the one-loop ghost-gluon vertex are also obtained. They allow for a check of consistency with the Ward--Slavnov--Taylor identity.
We investigate Yang-Mills theories with arbitrary gauge group on $R^3\times S^1$, whose symmetry is spontaneously broken by the Wilson loop. We show that instantons are made of fundamental magnetic monopoles, each of which has a corresponding root in the extended Dynkin diagram. The number of constituent magnetic monopoles for a single instanton is the dual Coxeter number of the gauge group, which also accounts for the number of instanton zero modes. In addition, we show that there exists a novel type of the $S^1$ coordinate dependent magnetic monopole solutions in $G_2,F_4,E_8$.
This work presents the calculation of the relation between the bare coupling constant g_0 and the MSbar-renormalized coupling g_MS, g_0 = Z_g(g_0,aμ) g_MS, to 2 loops in perturbation theory, with fermions in an arbitrary representation of the gauge group SU(N). Our calculation is performed using overlap fermions and Wilson gluons, and the background field technique has been chosen for convenience. The corresponding results in the fundamental representation appear in our longer publication [arXiv:0709.4368]. The 3-loop coefficient of the bare beta-function, b_2^L, is extracted using the 2-loop …
The mass spectrum of pure Yang-Mills theory in 3+1 dimensions is discussed for an arbitrary simple gauge algebra within a quasigluon picture. The general structure of the low-lying gluelump and two-quasigluon glueball spectrum is shown to be common to all algebras, while the lightest $C=-$ three-quasigluon glueballs only exist when the gauge algebra is A$_{r\geq 2}$, that is in particular $\mathfrak{su}(N\geq3)$. Higher-lying $C=-$ glueballs are shown to exist only for the A$_{r\geq2}$, D$_{{\rm odd}-r\geq 4}$ and E$_6$ gauge algebras. The shape of the static energy between adjoint sources is …
We discuss how to implement, in lattice gauge theories, external charges which are not commensurate with an elementary gauge coupling. It is shown that an arbitrary, real power of a standard Wilson loop (or Polyakov line) can be defined and consistently computed in lattice formulation of non-abelian, two dimensional gauge theories. However, such an observable can excite quantum states with integer fluxes only. Since the non-integer fluxes are not in the spectrum of the theory they cannot be created, no matter which observable is chosen. Also the continuum limit of above averages does not exist…
We study the entanglement entropy (EE) for pure gauge theories in 1+1 dimensions with the lattice regularization. Using the definition of the EE for lattice gauge theories proposed in a previous paper [1] (S. Aoki, T. Iritani, M. Nozaki, T. Numasawa, N. Shiba and H. Tasaki, JHEP 1506 (2015) 187), we calculate the EE for arbitrary pure as well as mixed states in terms of eigenstates of the transfer matrix in 1+1 dimensional lattice gauge theory. We find that the EE of an arbitrary pure state does not depend on the lattice spacing, thus giving the EE in the continuum limit, and show that the EE …
We focus on the issue of proper definition of entanglement entropy in lattice gauge theories, and examine a naive definition where gauge invariant states are viewed as elements of an extended Hilbert space which contains gauge non-invariant states as well. Working in the extended Hilbert space, we can define entanglement entropy associated with an arbitrary subset of links, not only for abelian but also for non-abelian theories. We then derive the associated replica formula. We also discuss the issue of gauge invariance of the entanglement entropy. In the $Z_N$ gauge theories in arbitrary spac…
