首页/论文/2012.10885

arXiv 2012.108852020-12cs.LGstat.ML

LieTransformer: Equivariant self-attention for Lie Groups

University of Oxford（牛津大学）；DeepMind（深度思维）

Michael Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, Hyunjik Kim

英文摘要：

Group equivariant neural networks are used as building blocks of group invariant neural networks, which have been shown to improve generalisation performance and data efficiency through principled parameter sharing. Such works have mostly focused on group equivariant convolutions, building on the result that group equivariant linear maps are necessarily convolutions. In this work, we extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups. We demonstrate the generality of our approach by showing experimental results that are competitive to baseline methods on a wide range of tasks: shape counting on point clouds, molecular property regression and modelling particle trajectories under Hamiltonian dynamics.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

齐性空间上神经网络的等变非线性映射

Equivariant non-linear maps for neural networks on homogeneous spaces

Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）；Massachusetts Institute of Technology（麻省理工学院）

本文提出了齐性空间上非线性等变神经网络层的通用框架，推导了广义可控性约束并证明其普适性，为未来等变网络层设计提供指导，并展示了多种常见等变网络架构均可由该框架推导得出。

2025-04-29cs.LGmath.RTstat.ML

Computing Representations for Lie Algebraic Networks

Intel AI（英特尔人工智能）；Stanford University（斯坦福大学）；Google Quantum AI（谷歌量子人工智能）；Rockley Photonics Inc.（罗克利光子公司）

Recent work has constructed neural networks that are equivariant to continuous symmetry groups such as 2D and 3D rotations. This is accomplished using explicit Lie group representations to derive the equivariant kernels and nonlinearities. We present three contributions motivated by frontier applications of equivariance beyond rotations and translations. First, we relax the requirement for explicit Lie group representations with a novel algorithm that finds representations of arbitrary Lie groups given only the structure constants of the associated Lie algebra. Second, we provide a self-contai…

2022-12-07cs.LGstat.ML

Group Equivariant Conditional Neural Processes

The University of Tokyo（东京大学）；RIKEN AIP（理化学研究所人工智能项目中心）

We present the group equivariant conditional neural process (EquivCNP), a meta-learning method with permutation invariance in a data set as in conventional conditional neural processes (CNPs), and it also has transformation equivariance in data space. Incorporating group equivariance, such as rotation and scaling equivariance, provides a way to consider the symmetry of real-world data. We give a decomposition theorem for permutation-invariant and group-equivariant maps, which leads us to construct EquivCNPs with an infinite-dimensional latent space to handle group symmetries. In this paper, we…

2021-02-17cs.LGstat.ML

ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groups

EPFL（洛桑联邦理工学院）；UvA（阿姆斯特丹大学）

We introduce ChebLieNet, a group-equivariant method on (anisotropic) manifolds. Surfing on the success of graph- and group-based neural networks, we take advantage of the recent developments in the geometric deep learning field to derive a new approach to exploit any anisotropies in data. Via discrete approximations of Lie groups, we develop a graph neural network made of anisotropic convolutional layers (Chebyshev convolutions), spatial pooling and unpooling layers, and global pooling layers. Group equivariance is achieved via equivariant and invariant operators on graphs with anisotropic lef…

2021-11-23cs.LGstat.ML

李群胚与李代数胚等变卷积神经网络的理论方面

Theoretical Aspects of Lie Groupoid and Lie Algebroid Equivariant Convolutional Neural Networks

Department of Mathematics, University of Manitoba（曼尼托巴大学数学系）

本文引入李群胚等变神经网络作为拓扑范畴等变神经网络在可微情形的特化，证明其与李代数胚等变神经网络的等价性，并推广了群不变全局池化。

2026-07-24cs.LGmath.CTmath.DG

李神经元：用于半单李代数的伴随等变神经网络

Lie Neurons: Adjoint-Equivariant Neural Networks for Semisimple Lie Algebras

University of Michigan（密歇根大学）

本文提出一种作用于半单李代数的伴随等变神经网络，通过李括号层和几何通道混合层扩展Vector Neurons，并在多个李代数任务中验证了其广泛适用性与竞争力。

2024-06-06cs.AIcs.LG

更多相关论文收起更多论文

约化李群上等变神经网络的通用框架

A General Framework for Equivariant Neural Networks on Reductive Lie Groups

University of Cambridge（剑桥大学）；ENS Paris-Saclay（巴黎萨克雷高等师范学院）；Université Paris-Saclay（巴黎萨克雷大学）；Massachusetts Institute of Technology（麻省理工学院）；EIA University（EIA大学）；University of British Columbia（不列颠哥伦比亚大学）

本文提出适用于任意约化李群有限维表示的通用等变神经网络框架，推广ACE与MACE，并发布lie-nn库，在顶夸克标记和形状识别任务中验证其通用性。

2023-05-31cs.LGhep-thstat.ML

在李代数上的一般线性对称性下的等变神经网络

Equivariant Neural Networks for General Linear Symmetries on Lie Algebras

ReLNs是一种支持一般线性对称性的等变神经网络，通过引入非退化双线性形式，实现矩阵和李代数特征的稳定交互，适用于多种对称性任务。

2026-01-30cs.LGcs.NE

LieAugmenter: 通过可学习的增强发现对称性实现等变学习

LieAugmenter: Equivariant Learning by Discovering Symmetries with Learnable Augmentations

Technical University of Munich (School of CIT)（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；MIT (Department of EECS and CSAIL)（麻省理工学院）

LieAugmenter通过可学习增强发现任务相关连续对称性，实现等变学习，提升图像分类、N体动力学和分子性质预测性能，并提供对称性缺失的可解释签名。

2026-02-12cs.LG

李代数规范化：任意李群下的等变神经算子

Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups

University of Cambridge（剑桥大学）；Harvard University（哈佛大学）

提出李代数规范化（LieLAC）方法，仅利用对称群无穷小生成元将输入转换为规范形式，使预训练模型在任意李群下实现等变性，成功应用于图像分类与PDE求解。

2025-03-04cs.CVcs.LGcs.NA

李群等变量子图神经网络

Lie-Equivariant Quantum Graph Neural Networks

QuaTI；UNESP（圣保罗州立大学）；IFT（理论物理研究所）；University of Florida（佛罗里达大学）；University of Alabama（阿拉巴马大学）；University of Kansas（堪萨斯大学）

本文提出李群等变量子图神经网络Lie-EQGNN，利用洛伦兹群等变性实现夸克-胶子喷注判别，性能与经典最优模型LorentzNet相当，为LHC数据分析提供了可行的量子替代方案。

2024-11-22cs.LGhep-exhep-ph

Automatic Symmetry Discovery with Lie Algebra Convolutional Network

Northwestern University（西北大学）；Northeastern University（东北大学）；University of California San Diego（加州大学圣地亚哥分校）

Existing equivariant neural networks require prior knowledge of the symmetry group and discretization for continuous groups. We propose to work with Lie algebras (infinitesimal generators) instead of Lie groups. Our model, the Lie algebra convolutional network (L-conv) can automatically discover symmetries and does not require discretization of the group. We show that L-conv can serve as a building block to construct any group equivariant feedforward architecture. Both CNNs and Graph Convolutional Networks can be expressed as L-conv with appropriate groups. We discover direct connections betwe…

2021-11-01cs.AIcs.LGmath.GR