Modeling global geometric context while maintaining equivariance is crucial for accurate predictions in many fields such as biology, chemistry, or vision. Yet, this is challenging due to the computational demands of processing high-dimensional data at scale. Existing approaches such as equivariant self-attention or distance-based message passing, suffer from quadratic complexity with respect to sequence length, while localized methods sacrifice global information. Inspired by the recent success of state-space and long-convolutional models, in this work, we introduce SE(3)-Hyena operator, an equivariant long-convolutional model based on the Hyena operator. The SE(3)-Hyena captures global geometric context at sub-quadratic complexity while maintaining equivariance to rotations and translations. Evaluated on equivariant associative recall and n-body modeling, SE(3)-Hyena matches or outperforms equivariant self-attention while requiring significantly less memory and computational resources for long sequences. Our model processes the geometric context of 20k tokens x3.5 times faster than the equivariant transformer and allows x175 longer a context within the same memory budget.
Scalable Peptide Design via Memory-Efficient Equivariant Transformer
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Department of Chemistry, Tsinghua University（清华大学化学系）
Ophiuchus: Scalable Modeling of Protein Structures through Hierarchical Coarse-graining SO(3)-Equivariant Autoencoders
Center for Bits and Atoms（比特与原子中心）；MIT Media Lab, Molecular Machines（麻省理工学院媒体实验室，分子机器）；Atomic Architects（原子建筑师）；MIT Research Laboratory of Electronics（麻省理工学院电子研究实验室）
LunaTP: Enabling Efficient and Scalable Tensor Product for Equivariant MLIPs
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所处理器重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Spin-Weighted Spherical Harmonics Enable Complete and Scalable $\mathrm{E}(3)$-Equivariant Networks
Department of Computer Science and Engineering, Texas A&M University（德克萨斯农工大学计算机科学与工程系）；Lambda, Inc.（Lambda公司）；Department of Materials Science and Engineering, Texas A&M University（德克萨斯农工大学材料科学与工程系）；Department of Electrical and Computer Engineering, Texas A&M University（德克萨斯农工大学电气与计算机工程系）；Department of Physics and Astronomy, Texas A&M University（德克萨斯农工大学物理与天文学系）；Department of Mechanical Engineering, Texas A&M University（德克萨斯农工大学机械工程系）
Amsterdam Machine Learning Lab (AMLab), University of Amsterdam（阿姆斯特丹机器学习实验室（AMLab），阿姆斯特丹大学）；New Theory（新理论）；Department of Mathematics and Computer Science, Eindhoven University of Technology（数学与计算机科学系，埃因霍温理工大学）；Leiden Institute of Advanced Computer Science, Universiteit Leiden（莱顿高级计算机科学研究所，莱顿大学）
GLASS: Global Latent Aggregation with Slot-based Set Decoding for Scalable All-Atom Crystal Generation
Institute for Artificial Intelligence（人工智能研究所）；University of Stuttgart（斯图加特大学）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；NEC Labs Europe（NEC欧洲实验室）
