Machine learning force fields (MLFFs) have become essential for accurate and efficient atomistic modeling. Despite their high accuracy, most existing approaches rely on fixed angular expansions, limiting flexibility in weighting local geometric interactions. We introduce Modular Angular-Radial Attention (MARA), a module that extends spherical attention -- originally developed for SO(3) tasks -- to the molecular domain and SE(3), providing an efficient approximation of equivariant interactions. MARA operates directly on the angular and radial coordinates of neighboring atoms, enabling flexible, geometrically informed, and modular weighting of local environments. Unlike existing attention mechanisms in SE(3)-equivariant architectures, MARA can be integrated in a plug-and-play manner into models such as MACE without architectural modifications. Across molecular benchmarks, MARA improves energy and force predictions, reduces high-error events, and enhances robustness. These results demonstrate that continuous spherical attention is an effective and generalizable geometric operator that increases the expressiveness, stability, and reliability of atomistic models.
TU Berlin（柏林工业大学）；BIFOLD, Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所（BIFOLD））；Google Research（谷歌研究院）；Korea University（高丽大学）；Max Planck Institut für Informatik（马克斯·普朗克信息学研究所）
The application of machine learning methods in quantum chemistry has enabled the study of numerous chemical phenomena, which are computationally intractable with traditional ab-initio methods. However, some quantum mechanical properties of molecules and materials depend on non-local electronic effects, which are often neglected due to the difficulty of modeling them efficiently. This work proposes a modified attention mechanism adapted to the underlying physics, which allows to recover the relevant non-local effects. Namely, we introduce spherical harmonic coordinates (SPHCs) to reflect higher…
Sampling 3D Molecular Conformers with Diffusion Transformers
Technical University Berlin（技术大学柏林）；BIFOLD Berlin（柏林BIFOLD）；Google DeepMind（谷歌DeepMind）；MPI for Informatics, Saarbrücken（萨尔布吕肯信息研究所）；Department of Artificial Intelligence, Korea University（韩国大学人工智能系）
The prediction of quantum mechanical properties is historically plagued by a trade-off between accuracy and speed. Machine learning potentials have previously shown great success in this domain, reaching increasingly better accuracy while maintaining computational efficiency comparable with classical force fields. In this work we propose TorchMD-NET, a novel equivariant transformer (ET) architecture, outperforming state-of-the-art on MD17, ANI-1, and many QM9 targets in both accuracy and computational efficiency. Through an extensive attention weight analysis, we gain valuable insights into th…
Equivariant Masked Position Prediction for Efficient Molecular Representation
Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）；INFLY TECH (Shanghai) Co., Ltd.（影力科技（上海）有限公司）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）
PERM EQ x GRAPH EQ: Equivariant Neural Networks for Quantum Molecular Learning
Independent Researcher, College Park, Maryland, United States（独立研究者，马里兰州学院公园，美国）；Independent AI Researcher, San Francisco, California, United States（独立AI研究者，加利福尼亚州旧金山，美国）
Equivariant Asynchronous Diffusion: An Adaptive Denoising Schedule for Accelerated Molecular Conformation Generation
Shanghai Academy of Artificial Intelligence for Science, SAIS（上海人工智能科学研究院，SAIS）；College of Chemistry and Chemical Engineering, Xiamen University（厦门大学化学与化工学院）；Artificial Intelligence Innovation and Incubation (AI 3 ) Institute, Fudan University（复旦大学人工智能创新与孵化（AI 3）研究院）
Technische Universität Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
Message passing neural networks have become a method of choice for learning on graphs, in particular the prediction of chemical properties and the acceleration of molecular dynamics studies. While they readily scale to large training data sets, previous approaches have proven to be less data efficient than kernel methods. We identify limitations of invariant representations as a major reason and extend the message passing formulation to rotationally equivariant representations. On this basis, we propose the polarizable atom interaction neural network (PaiNN) and improve on common molecule benc…
Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Tsinghua University（清华大学）；Peking University（北京大学）
Deep learning has achieved tremendous success in designing novel chemical compounds with desirable pharmaceutical properties. In this work, we focus on a new type of drug design problem -- generating a small "linker" to physically attach two independent molecules with their distinct functions. The main computational challenges include: 1) the generation of linkers is conditional on the two given molecules, in contrast to generating full molecules from scratch in previous works; 2) linkers heavily depend on the anchor atoms of the two molecules to be connected, which are not known beforehand; 3…
Learning and reasoning about 3D molecular structures with varying size is an emerging and important challenge in machine learning and especially in drug discovery. Equivariant Graph Neural Networks (GNNs) can simultaneously leverage the geometric and relational detail of the problem domain and are known to learn expressive representations through the propagation of information between nodes leveraging higher-order representations to faithfully express the geometry of the data, such as directionality in their intermediate layers. In this work, we propose an equivariant GNN that operates with Ca…
Freie Universität Berlin（柏林自由大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；EPFL（洛桑联邦理工学院）；Rice University（莱斯大学）
Equivariant neural networks (ENNs) are graph neural networks embedded in $\mathbb{R}^3$ and are well suited for predicting molecular properties. The ENN library e3nn has customizable convolutions, which can be designed to depend only on distances between points, or also on angular features, making them rotationally invariant, or equivariant, respectively. This paper studies the practical value of including angular dependencies for molecular property prediction directly via an ablation study with \texttt{e3nn} and the QM9 data set. We find that, for fixed network depth and parameter count, addi…
Tsinghua University（清华大学）；Renmin University of China（中国人民大学）；Huawei（华为）；Tsinghua-Huawei Joint Center for AI（清华-华为联合人工智能中心）；BNRist Center（北京国家信息科学与技术研究中心）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Beijing Key Laboratory of Big Data Management and Analysis Methods（大数据管理与分析方法北京市重点实验室）
Inverse molecular design is critical in material science and drug discovery, where the generated molecules should satisfy certain desirable properties. In this paper, we propose equivariant energy-guided stochastic differential equations (EEGSDE), a flexible framework for controllable 3D molecule generation under the guidance of an energy function in diffusion models. Formally, we show that EEGSDE naturally exploits the geometric symmetry in 3D molecular conformation, as long as the energy function is invariant to orthogonal transformations. Empirically, under the guidance of designed energy f…
EPFL（洛桑联邦理工学院）；Massachusetts Institute of Technology（麻省理工学院）；Microsoft Research AI4Science（微软研究院AI4Science）；University of Oxford（牛津大学）
Fragment-based drug discovery has been an effective paradigm in early-stage drug development. An open challenge in this area is designing linkers between disconnected molecular fragments of interest to obtain chemically-relevant candidate drug molecules. In this work, we propose DiffLinker, an E(3)-equivariant 3D-conditional diffusion model for molecular linker design. Given a set of disconnected fragments, our model places missing atoms in between and designs a molecule incorporating all the initial fragments. Unlike previous approaches that are only able to connect pairs of molecular fragmen…
In this paper, we propose Multiresolution Equivariant Graph Variational Autoencoders (MGVAE), the first hierarchical generative model to learn and generate graphs in a multiresolution and equivariant manner. At each resolution level, MGVAE employs higher order message passing to encode the graph while learning to partition it into mutually exclusive clusters and coarsening into a lower resolution that eventually creates a hierarchy of latent distributions. MGVAE then constructs a hierarchical generative model to variationally decode into a hierarchy of coarsened graphs. Importantly, our propos…
Frame-based Equivariant Diffusion Models for 3D Molecular Generation
University of Amsterdam（阿姆斯特丹大学）；AMLab；AI4Science Lab（AI4Science实验室）；Korteweg-de Vries Institute for Mathematics University of Amsterdam（Korteweg-de Vries数学研究所阿姆斯特丹大学）
Tsinghua University（清华大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Renmin University of China（中国人民大学）；Tencent AI Lab（腾讯人工智能实验室）
Pretraining molecular representation models without labels is fundamental to various applications. Conventional methods mainly process 2D molecular graphs and focus solely on 2D tasks, making their pretrained models incapable of characterizing 3D geometry and thus defective for downstream 3D tasks. In this work, we tackle 3D molecular pretraining in a complete and novel sense. In particular, we first propose to adopt an equivariant energy-based model as the backbone for pretraining, which enjoys the merits of fulfilling the symmetry of 3D space. Then we develop a node-level pretraining loss fo…
This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree-Fock computations. A molecular-orbital-based (MOB) pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of molecular orbitals (MOs). The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant Gaussian pro…
Shape-based virtual screening is widely employed in ligand-based drug design to search chemical libraries for molecules with similar 3D shapes yet novel 2D chemical structures compared to known ligands. 3D deep generative models have the potential to automate this exploration of shape-conditioned 3D chemical space; however, no existing models can reliably generate valid drug-like molecules in conformations that adopt a specific shape such as a known binding pose. We introduce a new multimodal 3D generative model that enables shape-conditioned 3D molecular design by equivariantly encoding molec…
Institute for Artificial Intelligence, Peking University, Beijing, China（北京大学人工智能研究院）；School of Intelligence Science and Technology, Peking University, Beijing, China（北京大学智能科学与技术学院）；Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA（麻省理工学院电子工程与计算机科学系）；Department of Automation, Tsinghua University, Beijing, China（清华大学自动化系）
Enhancing molecular dynamics with equivariant machine-learned densities
BIFOLD and Machine Learning Group, Technische Universität Berlin（BIFOLD与机器学习组，柏林技术大学）；Department of Chemistry, New York University（纽约大学化学系）；Department of Artificial Intelligence, Korea University（韩国大学人工智能系）；Max-Planck-Institut für Informatik（马克斯·普朗克研究所（信息学））；Department of Physics and Astronomy, University of California, Irvine（加州大学 Irvine 分校物理与天文学系）；Department of Chemistry, University of California, Irvine（加州大学 Irvine 分校化学系）；Courant Institute of Mathematical Sciences, New York University（纽约大学数学科学学院）；NYU-ECNU Center for Computational Chemistry at NYU Shanghai（纽约大学上海计算化学中心）
An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learning
Tsinghua University（清华大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Renmin University of China（中国人民大学）；University of Pisa（比萨大学）
Harvard University（哈佛大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Robert Bosch Research and Technology Center（罗伯特·博世研究与技术中心）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Massachusetts Institute of Technology（麻省理工学院）
This work presents Neural Equivariant Interatomic Potentials (NequIP), an E(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations for molecular dynamics simulations. While most contemporary symmetry-aware models use invariant convolutions and only act on scalars, NequIP employs E(3)-equivariant convolutions for interactions of geometric tensors, resulting in a more information-rich and faithful representation of atomic environments. The method achieves state-of-the-art accuracy on a challenging and diverse set of molecules and materials while ex…
UvA-Bosch Delta Lab（阿姆斯特丹大学-博世德尔塔实验室）；University of Amsterdam（阿姆斯特丹大学）
This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E(n)-Equivariant Graph Neural Networks (EGNNs). In contrast with existing methods, our work does not require computationally expensive higher-order representations in intermediate layers while it still achieves competitive or better performance. In addition, whereas existing methods are limited to equivariance on 3 dimensional spaces, our model is easily scaled to higher-dimensional spaces. We demonstrate the effectiveness of our method on dynamical syste…
GeoMFormer: A General Architecture for Geometric Molecular Representation Learning
School of EECS, Peking University（北京大学工程科学学院）；National Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Microsoft Research AI4Science（微软研究院AI4Science部门）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）
Processing information on 3D objects requires methods stable to rigid-body transformations, in particular rotations, of the input data. In image processing tasks, convolutional neural networks achieve this property using rotation-equivariant operations. However, contrary to images, graphs generally have irregular topology. This makes it challenging to define a rotation-equivariant convolution operation on these structures. In this work, we propose Spherical Graph Convolutional Network (S-GCN) that processes 3D models of proteins represented as molecular graphs. In a protein molecule, individua…
