Graph neural networks have recently become a standard method for analysing chemical compounds. In the field of molecular property prediction, the emphasis is now put on designing new model architectures, and the importance of atom featurisation is oftentimes belittled. When contrasting two graph neural networks, the use of different atom features possibly leads to the incorrect attribution of the results to the network architecture. To provide a better understanding of this issue, we compare multiple atom representations for graph models and evaluate them on the prediction of free energy, solubility, and metabolic stability. To the best of our knowledge, this is the first methodological study that focuses on the relevance of atom representation to the predictive performance of graph neural networks.
University of Science and Technology of China（中国科学技术大学）；Tencent America（腾讯美国）
Molecular property prediction (e.g., energy) is an essential problem in chemistry and biology. Unfortunately, many supervised learning methods usually suffer from the problem of scarce labeled molecules in the chemical space, where such property labels are generally obtained by Density Functional Theory (DFT) calculation which is extremely computational costly. An effective solution is to incorporate the unlabeled molecules in a semi-supervised fashion. However, learning semi-supervised representation for large amounts of molecules is challenging, including the joint representation issue of bo…
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Molecule property prediction is a fundamental problem for computer-aided drug discovery and materials science. Quantum-chemical simulations such as density functional theory (DFT) have been widely used for calculating the molecule properties, however, because of the heavy computational cost, it is difficult to search a huge number of potential chemical compounds. Machine learning methods for molecular modeling are attractive alternatives, however, the development of expressive, accurate, and scalable graph neural networks for learning molecular representations is still challenging. In this wor…
Firmenich SA（芬美意公司）；Helmholtz Zentrum München（亥姆霍兹慕尼黑中心）；BIGCHEM GmbH（BIGCHEM有限公司）
Prediction of molecular properties, including physico-chemical properties, is a challenging task in chemistry. Herein we present a new state-of-the-art multitask prediction method based on existing graph neural network models. We have used different architectures for our models and the results clearly demonstrate that multitask learning can improve model performance. Additionally, a significant reduction of variance in the models has been observed. Most importantly, datasets with a small amount of data points reach better results without the need of augmentation.
Designing accurate deep learning models for molecular property prediction plays an increasingly essential role in drug and material discovery. Recently, due to the scarcity of labeled molecules, self-supervised learning methods for learning generalizable and transferable representations of molecular graphs have attracted lots of attention. In this paper, we argue that there exist two major issues hindering current self-supervised learning methods from obtaining desired performance on molecular property prediction, that is, the ill-defined pre-training tasks and the limited model capacity. To t…
University of Notre Dame（圣母大学）；Brandeis University（布兰迪斯大学）
The recent success of graph neural networks has significantly boosted molecular property prediction, advancing activities such as drug discovery. The existing deep neural network methods usually require large training dataset for each property, impairing their performances in cases (especially for new molecular properties) with a limited amount of experimental data, which are common in real situations. To this end, we propose Meta-MGNN, a novel model for few-shot molecular property prediction. Meta-MGNN applies molecular graph neural network to learn molecular representation and builds a meta-…
Graph neural networks for molecular property prediction are frequently underspecified by data and fail to generalise to new scaffolds at test time. A potential solution is Bayesian learning, which can capture our uncertainty in the model parameters. This study benchmarks a set of Bayesian methods applied to a directed MPNN, using the QM9 regression dataset. We find that capturing uncertainty in both readout and message passing parameters yields enhanced predictive accuracy, calibration, and performance on a downstream molecular search task.
Research into deep learning models for molecular property prediction has primarily focused on the development of better Graph Neural Network (GNN) architectures. Though new GNN variants continue to improve performance, their modifications share a common theme of alleviating problems intrinsic to their fundamental graph-to-graph nature. In this work, we examine these limitations and propose a new molecular representation that bypasses the need for GNNs entirely, dubbed CubeMol. Our fixed-dimensional stochastic representation, when paired with a transformer model, exceeds the performance of stat…
Recently, the study of graph neural network (GNN) has attracted much attention and achieved promising performance in molecular property prediction. Most GNNs for molecular property prediction are proposed based on the idea of learning the representations for the nodes by aggregating the information of their neighbor nodes (e.g. atoms). Then, the representations can be passed to subsequent layers to deal with individual downstream tasks. Therefore, the architectures of GNNs can be considered as being composed of two core parts: graph-related layers and task-specific layers. Facing real-world mo…
In this work, we propose the novel Prototypical Graph Regression Self-explainable Trees (ProGReST) model, which combines prototype learning, soft decision trees, and Graph Neural Networks. In contrast to other works, our model can be used to address various challenging tasks, including compound property prediction. In ProGReST, the rationale is obtained along with prediction due to the model's built-in interpretability. Additionally, we introduce a new graph prototype projection to accelerate model training. Finally, we evaluate PRoGReST on a wide range of chemical datasets for molecular prope…
Previous works have demonstrated the importance of considering different modalities on molecules, each of which provide a varied granularity of information for downstream property prediction tasks. Our method combines variants of the recent TransformerM architecture with Transformer, GNN, and ResNet backbone architectures. Models are trained on the 2D data, 3D data, and image modalities of molecular graphs. We ensemble these models with a HuberRegressor. The models are trained on 4 different train/validation splits of the original train + valid datasets. This yields a winning solution to the 2…
MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction
New York University Shanghai（上海纽约大学）；Xiamen University（厦门大学）；University of California, San Francisco（加利福尼亚大学旧金山分校）；Zhejiang University（浙江大学）；The Hong Kong Polytechnic University（香港理工大学）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Cornell University（康奈尔大学）
We demonstrate the applicability of model-agnostic algorithms for meta-learning, specifically Reptile, to GNN models in molecular regression tasks. Using meta-learning we are able to learn new chemical prediction tasks with only a few model updates, as compared to using randomly initialized GNNs which require learning each regression task from scratch. We experimentally show that GNN layer expressivity is correlated to improved meta-learning. Additionally, we also experiment with GNN emsembles which yield best performance and rapid convergence for k-shot learning.
SPECTRA: Spectral Domain-Aware Graph Generation for Imbalanced Molecular Property Regression
University of Notre Dame, Dept. of Computer Science；University of Notre Dame, Dept. of Chemistry；University of Notre Dame, Lucy Family Institute for Data \& Society Notre Dame Indiana USA；University of Notre Dame, Lucy Family Institute for Data \& Society
Deep learning is an important method for molecular design and exhibits considerable ability to predict molecular properties, including physicochemical, bioactive, and ADME/T (absorption, distribution, metabolism, excretion, and toxicity) properties. In this study, we advanced a novel deep learning architecture, termed FP-GNN, which combined and simultaneously learned information from molecular graphs and fingerprints. To evaluate the FP-GNN model, we conducted experiments on 13 public datasets, an unbiased LIT-PCBA dataset, and 14 phenotypic screening datasets for breast cell lines. Extensive …
Graph neural networks (GNNs) have been proposed for a wide range of graph-related learning tasks. In particular, in recent years, an increasing number of GNN systems were applied to predict molecular properties. However, a direct impediment is to select appropriate hyperparameters to achieve satisfactory performance with lower computational cost. Meanwhile, many molecular datasets are far smaller than many other datasets in typical deep learning applications. Most hyperparameter optimization (HPO) methods have not been explored in terms of their efficiencies on such small datasets in the molec…
KAIST（韩国科学技术院）；Seoul National University（首尔国立大学）；KAIST Institute for Artificial Intelligence（韩国科学技术院人工智能研究所）
In chemistry, deep neural network models have been increasingly utilized in a variety of applications such as molecular property predictions, novel molecule designs, and planning chemical reactions. Despite the rapid increase in the use of state-of-the-art models and algorithms, deep neural network models often produce poor predictions in real applications because model performance is highly dependent on the quality of training data. In the field of molecular analysis, data are mostly obtained from either complicated chemical experiments or approximate mathematical equations, and then quality …
Research Institute of Sweden (RISE)（瑞典研究院（RISE））；Uppsala University（乌普萨拉大学）
Molecular property prediction is essential in chemistry, especially for drug discovery applications. However, available molecular property data is often limited, encouraging the transfer of information from related data. Transfer learning has had a tremendous impact in fields like Computer Vision and Natural Language Processing signaling for its potential in molecular property prediction. We present a pre-training procedure for molecular representation learning using reaction data and use it to pre-train a SMILES Transformer. We fine-tune and evaluate the pre-trained model on 12 molecular prop…
Molecular Representation Learning is essential to solving many drug discovery and computational chemistry problems. It is a challenging problem due to the complex structure of molecules and the vast chemical space. Graph representations of molecules are more expressive than traditional representations, such as molecular fingerprints. Therefore, they can improve the performance of machine learning models. We propose SeMole, a method that augments the Junction Tree Variational Autoencoders, a state-of-the-art generative model for molecular graphs, with semi-supervised learning. SeMole aims to im…
Seoul National University（首尔大学）；Kakao Corporation（Kakao公司）；Seoul National University of Science and Technology（首尔科学技术院）
A molecule is a complex of heterogeneous components, and the spatial arrangements of these components determine the whole molecular properties and characteristics. With the advent of deep learning in computational chemistry, several studies have focused on how to predict molecular properties based on molecular configurations. Message passing neural network provides an effective framework for capturing molecular geometric features with the perspective of a molecule as a graph. However, most of these studies assumed that all heterogeneous molecular features, such as atomic charge, bond length, o…
In this report, we (SuperHelix team) present our solution to KDD Cup 2021-PCQM4M-LSC, a large-scale quantum chemistry dataset on predicting HOMO-LUMO gap of molecules. Our solution, Lite Geometry Enhanced Molecular representation learning (LiteGEM) achieves a mean absolute error (MAE) of 0.1204 on the test set with the help of deep graph neural networks and various self-supervised learning tasks. The code of the framework can be found in https://github.com/PaddlePaddle/PaddleHelix/tree/dev/competition/kddcup2021-PCQM4M-LSC/.
MolKD: Distilling Cross-Modal Knowledge in Chemical Reactions for Molecular Property Prediction
Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University（清华大学交叉信息研究院）；Zhejiang Lab（之江实验室）；The Chinese University of Hong Kong（香港中文大学）
University of Toronto（多伦多大学）；Reverie Labs；DeepChem
GNNs and chemical fingerprints are the predominant approaches to representing molecules for property prediction. However, in NLP, transformers have become the de-facto standard for representation learning thanks to their strong downstream task transfer. In parallel, the software ecosystem around transformers is maturing rapidly, with libraries like HuggingFace and BertViz enabling streamlined training and introspection. In this work, we make one of the first attempts to systematically evaluate transformers on molecular property prediction tasks via our ChemBERTa model. ChemBERTa scales well wi…
We show that the pressure acting on atoms and molecular systems within the compression cavity of the eXtreme-Pressure Polarizable Continuum method can be expressed in terms of the electron density of the systems and of the Pauli-repulsion confining potential. The analytical expression holds for spherical cavities as well as for cavities constructed from van der Waals spheres of the constituting atoms of the molecular systems.
Machine learning holds the promise of learning the energy functional via examples, bypassing the need to solve complicated quantum-chemical equations and realizing efficient computing of molecular electronic properties.
Atomic effective one-electron potentials in a compact analytic form in terms of a few Gaussian charge distributions are developed, for Hydrogen through Nobelium, for starting molecular electronic structure calculations by a simple diagonalization. For each element, all terms but one are optimized in an isolated-atom Hartree--Fock calculation, and the last one is parametrized on a set of molecules. This one-parameter-per-atom model gives a good starting guess for typical molecules and may be of interest even on its own.
AtomDisc: An Atom-level Tokenizer that Boosts Molecular LLMs and Reveals Structure--Property Associations
The Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（人工智能动力部，香港科学与技术大学（广州））；The College of Computer Science and Technology, The Nanjing University of Aeronautics and Astronautics（计算机科学与技术学院，南京航空航天大学）
Balancing property optimization and constraint satisfaction for constrained multi-property molecular optimization
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与电子工程学院）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）
We report on a study of cold reactive collisions between sympathetically-cooled molecular ions and laser-cooled atoms in an ion-atom hybrid trap. Chemical reactions were studied at average collision energies <Ecoll>/k > 20 mK, about two orders of magnitude lower than has been achieved in previous experiments with molecular ions. Choosing N2+ +Rb as a prototypical system, we find that the reaction rate is independent of the collision energy, but strongly dependent on the internal state of Rb. Highly efficient charge exchange about four times faster than the Langevin rate was observed with Rb in…
We study the 38-atom Lennard-Jones cluster with parallel tempering Monte Carlo methods in the microcanonical and molecular dynamics ensembles. A new Monte Carlo algorithm is presented that samples rigorously the molecular dynamics ensemble for a system at constant total energy, linear and angular momenta. By combining the parallel tempering technique with molecular dynamics methods, we develop a hybrid method to overcome quasi-ergodicity and to extract both equilibrium and dynamical properties from Monte Carlo and molecular dynamics simulations. Several thermodynamic, structural and dynamical …
We present a novel binding mechanism where a neutral Rydberg atom and an atomic ion form a molecular bound state at large internuclear distance. The binding mechanism is based on Stark shifts and level crossings which are induced in the Rydberg atom due to the electric field of the ion. At particular internuclear distances between Rydberg atom and ion, potential wells occur which can hold atom-ion molecular bound states. Apart from the binding mechanism we describe important properties of the long-range atom-ion Rydberg molecule, such as its lifetime and decay paths, its vibrational and rotati…
A double-index atomic partitioning of the molecular first-order density matrix is proposed. Contributions diagonal in the atomic indices correspond to atomic density matrices, whereas off-diagonal contributions carry information about the bonds. The resulting matrices have good localization properties, in contrast to single-index atomic partitioning schemes of the molecular density matrix. It is shown that the electron density assigned to individual atoms, when derived from the density matrix partitioning, can be made con- sistent with well-known partitions of the electron density over AIM bas…
We demonstrate full quantum state control of two species of single atoms using optical tweezers and assemble the atoms into a molecule. Our demonstration includes 3D ground-state cooling of a single atom (Cs) in an optical tweezer, transport by several microns with minimal heating, and merging with a single Na atom. Subsequently, both atoms occupy the simultaneous motional ground state with 61(4)\% probability. This realizes a sample of exactly two co-trapped atoms near the phase-space-density limit of one, and allows for efficient stimulated-Raman transfer of a pair of atoms into a molecular …
We describe methods for the production of spin-polarized H and D atoms from the IR-excitation and photodissociation of molecular beams of HBr, HI, and ${\rm NH_{3}}$ isotopes, including optical excitation schemes with partial hyperfine resolution. We discuss the extent to which the production rates may approach the IR-laser production rates of ${\rm 10^{21}\, photons\, s^{-1}}$, and how the production rates of conventional methods of ${\sim}{\rm 10^{17} \, s^{-1}}$ may be surpassed significantly.
