Our universe is homogeneous and isotropic, and its perturbations obey translation and rotation symmetry. In this work we develop Translation and Rotation Equivariant Normalizing Flow (TRENF), a generative Normalizing Flow (NF) model which explicitly incorporates these symmetries, defining the data likelihood via a sequence of Fourier space-based convolutions and pixel-wise nonlinear transforms. TRENF gives direct access to the high dimensional data likelihood p(x|y) as a function of the labels y, such as cosmological parameters. In contrast to traditional analyses based on summary statistics, the NF approach has no loss of information since it preserves the full dimensionality of the data. On Gaussian random fields, the TRENF likelihood agrees well with the analytical expression and saturates the Fisher information content in the labels y. On nonlinear cosmological overdensity fields from N-body simulations, TRENF leads to significant improvements in constraining power over the standard power spectrum summary statistic. TRENF is also a generative model of the data, and we show that TRENF samples agree well with the N-body simulations it trained on, and that the inverse mapping of the data agrees well with a Gaussian white noise both visually and on various summary statistics: when this is perfectly achieved the resulting p(x|y) likelihood analysis becomes optimal. Finally, we develop a generalization of this model that can handle effects that break the symmetry of the data, such as the survey mask, which enables likelihood analysis on data without periodic boundaries.
UvA-Bosch Delta Lab（阿姆斯特丹大学-博世三角洲实验室）；University of Amsterdam（阿姆斯特丹大学）；University of Oxford（牛津大学）
This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs). To construct E-NFs, we take the discriminative E(n) graph neural networks and integrate them as a differential equation to obtain an invertible equivariant function: a continuous-time normalizing flow. We demonstrate that E-NFs considerably outperform baselines and existing methods from the literature on particle systems such as DW4 and LJ13, and on molecules from QM9 in terms of log-likelihood. To the best of our knowledge, this is the first flow that jointly generates mol…
Generative modeling seeks to uncover the underlying factors that give rise to observed data that can often be modeled as the natural symmetries that manifest themselves through invariances and equivariances to certain transformation laws. However, current approaches to representing these symmetries are couched in the formalism of continuous normalizing flows that require the construction of equivariant vector fields -- inhibiting their simple application to conventional higher dimensional generative modelling domains like natural images. In this paper, we focus on building equivariant normaliz…
We propose an algorithm for learning a conditional generative model of a molecule given a target. Specifically, given a receptor molecule that one wishes to bind to, the conditional model generates candidate ligand molecules that may bind to it. The distribution should be invariant to rigid body transformations that act $\textit{jointly}$ on the ligand and the receptor; it should also be invariant to permutations of either the ligand or receptor atoms. Our learning algorithm is based on a continuous normalizing flow. We establish semi-equivariance conditions on the flow which guarantee the afo…
Simulating the Hubbard Model with Equivariant Normalizing Flows
University of Bonn（波恩大学）；Institute for Advanced Simulation 4 (IAS-4)（高级仿真研究所4）；Center for Advanced Simulation and Analytics (CASA)（高级仿真与分析中心）；University of the Virgin Islands（美属维尔京群岛大学）；College of Science and Mathematics（科学与数学学院）
JFlow: Model-Independent Spherical Jeans Analysis using Equivariant Continuous Normalizing Flows
Particle Theory and Cosmology Group, Center for Theoretical Physics of the Universe, Institute for Basic Science (IBS)（基本科学研究院（IBS）粒子理论与宇宙学组，宇宙理论物理中心）；Department of Physics and Astronomy, Rutgers, The State University of New Jersey（新泽西州立大学罗格斯大学物理与天文学系）；National Institute of Technology, Sendai College（Sendai学院国立技术研究所）；Astronomical Institute, Tohoku University（东北大学天文研究所）；ICRR, The University of Tokyo（东京大学ICRR）；Kavli Institute for the Physics and Mathematics of the Universe (WPI), The University of Tokyo Institutes for Advanced Study, The University of Tokyo（东京大学Kavli研究所（WPI），东京大学先进研究所，东京大学）；Theory Center, IPNS, KEK（IPNS理论中心，KEK）；Graduate University for Advanced Studies (SOKENDAI)（高级研究大学（SOKENDAI））
Freie Universität Berlin（柏林自由大学）；Rice University（莱斯大学）
Normalizing flows are exact-likelihood generative neural networks which approximately transform samples from a simple prior distribution to samples of the probability distribution of interest. Recent work showed that such generative models can be utilized in statistical mechanics to sample equilibrium states of many-body systems in physics and chemistry. To scale and generalize these results, it is essential that the natural symmetries in the probability density -- in physics defined by the invariances of the target potential -- are built into the flow. We provide a theoretical sufficient crit…
SESaMo: Symmetry-Enforcing Stochastic Modulation for Normalizing Flows
University of Bonn（波恩大学）；Helmholtz Institute for Radiation and Nuclear Physics (HISKP)（亥姆霍兹辐射与核物理研究所（HISKP））；Oldendorff Carriers GmbH & Co. KG（奥尔登多夫航运有限两合公司）
Modeling sets is an important problem in machine learning since this type of data can be found in many domains. A promising approach defines a family of permutation invariant densities with continuous normalizing flows. This allows us to maximize the likelihood directly and sample new realizations with ease. In this work, we demonstrate how calculating the trace, a crucial step in this method, raises issues that occur both during training and inference, limiting its practicality. We propose an alternative way of defining permutation equivariant transformations that give closed form trace. This…
Normalizing flows for lattice gauge theory in arbitrary space-time dimension
Massachusetts Institute of Technology（麻省理工学院）；New York University（纽约大学）；DeepMind（深度思维）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Bern（伯尔尼大学）
TU Wien（维也纳工业大学）；Institute for Theoretical Physics（理论物理研究所）
The introduction of relevant physical information into neural network architectures has become a widely used and successful strategy for improving their performance. In lattice gauge theories, such information can be identified with gauge symmetries, which are incorporated into the network layers of our recently proposed Lattice Gauge Equivariant Convolutional Neural Networks (L-CNNs). L-CNNs can generalize better to differently sized lattices than traditional neural networks and are by construction equivariant under lattice gauge transformations. In these proceedings, we present our progress …
Qualcomm AI Research（高通AI研究）；Qualcomm Technologies Netherlands B.V.（高通荷兰技术有限公司）；University of Amsterdam（阿姆斯特丹大学）
We propose a continuous normalizing flow for sampling from the high-dimensional probability distributions of Quantum Field Theories in Physics. In contrast to the deep architectures used so far for this task, our proposal is based on a shallow design and incorporates the symmetries of the problem. We test our model on the $ϕ^4$ theory, showing that it systematically outperforms a realNVP baseline in sampling efficiency, with the difference between the two increasing for larger lattices. On the largest lattice we consider, of size $32\times 32$, we improve a key metric, the effective sample siz…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck Institute for Gravitational Physics（马克斯·普朗克引力物理研究所）；University of Tübingen（图宾根大学）
Non-Perturbative Trivializing Flows for Lattice Gauge Theories
Institute of Physics, University of Amsterdam（阿姆斯特丹大学物理研究所）；Department of Computing, Imperial College London（伦敦帝国学院计算机系）；Institute for Mathematics, Academia Sinica, Taiwan（台湾“中央研究院”数学研究所）；Korteweg-de Vries Institute for Mathematics, University of Amsterdam（阿姆斯特丹大学柯尔特韦格-德维尔斯数学研究所）
Department of Mathematics and Mathematical Statistics（数学与统计学系）；Umeå University（乌梅大学）；Department of Mathematical Sciences（数学科学系）；Chalmers University of Technology and Gothenburg University（楚姆勒技术大学和哥德堡大学）
Inverted AI（Inverted AI公司）；University of British Columbia（不列颠哥伦比亚大学）；Mila（米拉研究所）
We present a novel, conditional generative probabilistic model of set-valued data with a tractable log density. This model is a continuous normalizing flow governed by permutation equivariant dynamics. These dynamics are driven by a learnable per-set-element term and pairwise interactions, both parametrized by deep neural networks. We illustrate the utility of this model via applications including (1) complex traffic scene generation conditioned on visually specified map information, and (2) object bounding box generation conditioned directly on images. We train our model by maximizing the exp…
A scalable flow-based approach to mitigate topological freezing
Albert Einstein Center for Fundamental Physics, Institute for Theoretical Physics, University of Bern（阿尔伯特·爱因斯坦基础物理学研究中心，理论物理学研究所，伯尔尼大学）；Higgs Centre for Theoretical Physics, School of Physics and Astronomy, The University of Edinburgh（希格斯理论物理学中心，物理与天文学学院，爱丁堡大学）；Centre for Mathematical Sciences, University of Plymouth（数学科学中心，普利茅斯大学）
