Equivariant Graph Neural Networks (EGNNs) have demonstrated significant success in modeling microscale systems, including those in chemistry, biology and materials science. However, EGNNs face substantial computational challenges due to the high cost of constructing edge features via spherical tensor products, making them impractical for large-scale systems. To address this limitation, we introduce E2Former, an equivariant and efficient transformer architecture that incorporates the Wigner $6j$ convolution (Wigner $6j$ Conv). By shifting the computational burden from edges to nodes, the Wigner $6j$ Conv reduces the complexity from $O(|\mathcal{E}|)$ to $ O(| \mathcal{V}|)$ while preserving both the model's expressive power and rotational equivariance. We show that this approach achieves a 7x-30x speedup compared to conventional $\mathrm{SO}(3)$ convolutions. Furthermore, our empirical results demonstrate that the derived E2Former mitigates the computational challenges of existing approaches without compromising the ability to capture detailed geometric information. This development could suggest a promising direction for scalable and efficient molecular modeling.
Constructing 3D Rotational Invariance and Equivariance with Symmetric Tensor Networks
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究所);School of Information Science and Technology, University of Science and Technology of China(信息科学与技术学院);CAS Key Laboratory of Quantum Information, University of Science and Technology of China(中国科学技术大学量子信息关键实验室);Hefei National Laboratory, University of Science and Technology of China(合肥国家实验室)
Equivariant geometric learning for digital rock physics: estimating formation factor and effective permeability tensors from Morse graph
Stony Brook University(石溪大学);Halicioglu Data Science Institute, University of California, San Diego(加州大学圣地亚哥分校哈利西奥格鲁数据科学研究所);Columbia University(哥伦比亚大学)
High-Rank Irreducible Cartesian Tensor Decomposition and Bases of Equivariant Spaces
State Key Lab of Vascular Homeostasis and Remodeling(血管稳态与重塑国家重点实验室);Department of Bioinformatics(生物信息学系);School of Basic Medical Sciences, Peking University(北京大学基础医学学院);State Key Lab for General AI(通用人工智能国家重点实验室);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);Peking University, China(北京大学,中国);Pazhou Laboratory (Huangpu), China(琶洲实验室(黄埔),中国)
P-Tensors: a General Formalism for Constructing Higher Order Message Passing Networks
Department of Computer Science(计算机科学系);University of Chicago(芝加哥大学);Computational and Applied Math Department of Statistics(计算与应用数学统计系);Department of Statistics(统计系)
On the Expressive Power of Floating-Point Transformers
Department of Artificial Intelligence, Korea University(韩国大学人工智能系);Department of Mathematics and Statistics, Sejong University(世宗大学数学与统计学系);Department of Mathematical Sciences, Gwangju Institute of Science and Technology(全州科学技术研究院数学科学系)
Scalable Peptide Design via Memory-Efficient Equivariant Transformer
Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);Department of Chemistry, Tsinghua University(清华大学化学系)
FlashCart: Fast Cartesian Tensor Products for Equivariant Interatomic Potentials
INM – Leibniz Institute for New Materials(莱布尼茨新材料研究所);Saarland University(萨尔兰大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);University of Stuttgart(斯图加特大学);NEC Laboratories Europe(NEC欧洲实验室)
Equivariant Covariance Tensors: Guaranteed SPD Uncertainty for Tensor-Valued Geometric Learning
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);School of Microelectronics, Fudan University(复旦大学微电子学院);ByteDance(字节跳动);School of Information Science and Engineering, East China University of Science and Technology(华东理工大学信息科学与工程学院)
International Professional University of Technology(国际技术职业大学);Brookhaven National Laboratory(布鲁克海文国家实验室);RIKEN Center for Advanced Intelligence Project (AIP)(理化学研究所先进智能项目中心)
D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation
School of Physics and Electronic Engineering, Jiangsu University(江苏大学物理与电子工程学院);Jiangsu Engineering Research Center on Quantum Perception and Intelligent Detection of Agricultural Information(江苏省农业信息量子感知与智能检测工程研究中心)
The Price of Freedom: Exploring Expressivity and Runtime Tradeoffs in Equivariant Tensor Products
Department of EECS, Massachusetts Institute of Technology, Cambridge MA 02139, USA(麻省理工学院电子工程与计算机科学系);Department of CSE, Massachusetts Institute of Technology, Cambridge MA 02139, USA(麻省理工学院计算机科学与工程系)