Optimization Dynamics of Equivariant and Augmented Neural Networks
Umeå University（于默奥大学）
Oskar Nordenfors, Fredrik Ohlsson, Axel Flinth
备注：
v4: Some discussions added, along with an updated experiment section. v3: Completely revised manuscript: New framework for neural nets, new main result (involving compability condition), new experiments, new author. v2: Revised manuscript. Mostly small edits, apart from new experiments (see Appendix E)
We investigate the optimization of neural networks on symmetric data, and compare the strategy of constraining the architecture to be equivariant to that of using data augmentation. Our analysis reveals that that the relative geometry of the admissible and the equivariant layers, respectively, plays a key role. Under natural assumptions on the data, network, loss, and group of symmetries, we show that compatibility of the spaces of admissible layers and equivariant layers, in the sense that the corresponding orthogonal projections commute, implies that the sets of equivariant stationary points are identical for the two strategies. If the linear layers of the network also are given a unitary parametrization, the set of equivariant layers is even invariant under the gradient flow for augmented models. Our analysis however also reveals that even in the latter situation, stationary points may be unstable for augmented training although they are stable for the manifestly equivariant models.
Learning (Approximately) Equivariant Networks via Constrained Optimization
Computer Science Department, University of Stuttgart（斯图加特大学计算机科学系）；International Max Planck Research School for Intelligent Systems（国际马克斯·普朗克智能系统研究学校）；Department of Applied Mathematics, École Polytechnique de Paris（巴黎高等理工学院应用数学系）
School of Management, Technical University of Munich, Germany（慕尼黑技术大学管理学院）；Munich Data Science Institute, Technical University of Munich, Germany（慕尼黑数据科学研究所）；Inria, Université de Lille, France（法国里尔大学Inria）；CERMICS, Ecole Nationale des Ponts et Chaussées, Institut Polytechnique Paris, France（巴黎高等理工学院CERMICS）
Universally Invariant Learning in Equivariant GNNs
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学明德学院）；Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大模型与智能治理重点实验室）；Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE（下一代智能搜索与推荐工程技术研究中心）；DAMO Academy, Alibaba Group（阿里达摩院）；Hupan Lab（虎斑实验室）
Equivariant Filters for Efficient Tracking in 3D Imaging
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；CSAIL, Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）；Boston Children’s Hospital（波士顿儿童医院）
LieAugmenter: Equivariant Learning by Discovering Symmetries with Learnable Augmentations
Technical University of Munich (School of CIT)（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；MIT (Department of EECS and CSAIL)（麻省理工学院）
Multi-objective optimization and quantum hybridization of equivariant deep learning interatomic potentials
Terra Quantum AG；LIACS, Leiden University（LIACS，莱顿大学）；Nanoscience Center and Department of Chemistry, University of Jyväskylä（贾瓦尔基利亚大学纳米科学中心和化学系）；Department of Materials Engineering, Indian Institute of Science（印度科学研究所材料工程系）；Schaeffler Technologies AG & Co. KG
Antibody Design and Optimization with Multi-scale Equivariant Graph Diffusion Models for Accurate Complex Antigen Binding
School of Computer Science, Wuhan University, Wuhan, China（武汉大学计算机学院）；Department of Computing, Macquarie University, Sydney, Australia（麦考瑞大学计算机系）；Wuhan University Shenzhen Research Institute, Shenzhen, China（武汉大学深圳研究院）
Multi-Response Preference Optimization with Augmented Ranking Dataset
INMED DATA；Yonsei University（延世大学）；Asan Medical Center（峨山医院）；University of Ulsan College of Medicine（蔚山大学医学院）；Asan Institute for Life Sciences（峨山生命科学研究所）
A distributed semismooth Newton based augmented Lagrangian method for distributed optimization
School of Mathematics, Southwest Jiaotong University（西南交通大学数学学院）；School of Computer and Computing Science, Hangzhou City University（杭州城市大学计算机与计算科学学院）；Institute of Mathematics, Zhejiang Wanli University（浙江万里大学数学学院）
Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketching
University of Chicago（芝加哥大学）；University of California, Berkeley（加州大学伯克利分校）；International Computer Science Institute（国际计算机科学研究所）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
Manipulating 3D Molecules in a Fixed-Dimensional E(3)-Equivariant Latent Space
Institute for AI Industry Research (AIR), Tsinghua University（人工智能产业研究院（AIR），清华大学）；Department of Computer Science and Technology, Tsinghua University（计算机科学与技术系，清华大学）；Qiuzhen College, Tsinghua University（启真学院，清华大学）
Clemson University（克莱姆森大学）；Université de Montréal（蒙特利尔大学）；Accenture (United States)（埃森哲（美国））；Arizona State University（亚利桑那州立大学）；Northeastern University（东北大学）
Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks
Eindhoven University of Technology（埃因霍温理工大学）；SISSA (International School for Advanced Studies)（国际高等研究院（SISSA））；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Center for Nonlinear Studies (CNLS)（非线性研究中心（CNLS））；Eindhoven Artificial Intelligence Systems Institute（埃因霍温人工智能系统研究所）
