Incorporating symmetry as an inductive bias into neural network architecture has led to improvements in generalization, data efficiency, and physical consistency in dynamics modeling. Methods such as CNNs or equivariant neural networks use weight tying to enforce symmetries such as shift invariance or rotational equivariance. However, despite the fact that physical laws obey many symmetries, real-world dynamical data rarely conforms to strict mathematical symmetry either due to noisy or incomplete data or to symmetry breaking features in the underlying dynamical system. We explore approximately equivariant networks which are biased towards preserving symmetry but are not strictly constrained to do so. By relaxing equivariance constraints, we find that our models can outperform both baselines with no symmetry bias and baselines with overly strict symmetry in both simulated turbulence domains and real-world multi-stream jet flow.
Learning (Approximately) Equivariant Networks via Constrained Optimization
Computer Science Department, University of Stuttgart（斯图加特大学计算机科学系）；International Max Planck Research School for Intelligent Systems（国际马克斯·普朗克智能系统研究学校）；Department of Applied Mathematics, École Polytechnique de Paris（巴黎高等理工学院应用数学系）
LieAugmenter: Equivariant Learning by Discovering Symmetries with Learnable Augmentations
Technical University of Munich (School of CIT)（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；MIT (Department of EECS and CSAIL)（麻省理工学院）
Recent research has shown that incorporating equivariance into neural network architectures is very helpful, and there have been some works investigating the equivariance of networks under group actions. However, as digital images and feature maps are on the discrete meshgrid, corresponding equivariance-preserving transformation groups are very limited. In this work, we deal with this issue from the connection between convolutions and partial differential operators (PDOs). In theory, assuming inputs to be smooth, we transform PDOs and propose a system which is equivariant to a much more genera…
UvA-Bosch Delta Lab（阿姆斯特丹大学-博世三角洲实验室）；University of Amsterdam（阿姆斯特丹大学）
In this work we seek to bridge the concepts of topographic organization and equivariance in neural networks. To accomplish this, we introduce the Topographic VAE: a novel method for efficiently training deep generative models with topographically organized latent variables. We show that such a model indeed learns to organize its activations according to salient characteristics such as digit class, width, and style on MNIST. Furthermore, through topographic organization over time (i.e. temporal coherence), we demonstrate how predefined latent space transformation operators can be encouraged for…
Freie Universität Berlin（柏林自由大学）；Rice University（莱斯大学）
Normalizing flows are exact-likelihood generative neural networks which approximately transform samples from a simple prior distribution to samples of the probability distribution of interest. Recent work showed that such generative models can be utilized in statistical mechanics to sample equilibrium states of many-body systems in physics and chemistry. To scale and generalize these results, it is essential that the natural symmetries in the probability density -- in physics defined by the invariances of the target potential -- are built into the flow. We provide a theoretical sufficient crit…
Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations
Department of Computer Science and Engineering, Texas A&M University, USA（计算机科学与工程系，德克萨斯A&M大学）；Lambda, Inc., USA（Lambda公司）；RIKEN Cluster for Pioneering Research, RIKEN, Japan（日本RIKEN先锋研究中心）；NVIDIA, USA（NVIDIA公司）；Department of Materials Science and Engineering, Texas A&M University, USA（材料科学与工程系，德克萨斯A&M大学）；Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, USA（机械工程系，德克萨斯A&M大学）
Probing the effects of broken symmetries in machine learning
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；National Centre for Computational Design and Discovery of Novel Materials MARVEL（新型材料计算设计与发现国家中心MARVEL）；Institute of Materials（材料研究所）；Laboratory of Computational Science and Modeling（计算科学与建模实验室）
