Since the advent of machine learning, interpretability has remained a persistent challenge, becoming increasingly urgent as generative models support high-stakes applications in drug and material discovery. Recent advances in large language model (LLM) architectures have yielded chemistry language models (CLMs) with impressive capabilities in molecular property prediction and molecular generation. However, how these models internally represent chemical knowledge remains poorly understood. In this work, we extend sparse autoencoder techniques to uncover and examine interpretable features within CLMs. Applying our methodology to the Foundation Models for Materials (FM4M) SMI-TED chemistry foundation model, we extract semantically meaningful latent features and analyse their activation patterns across diverse molecular datasets. Our findings reveal that these models encode a rich landscape of chemical concepts. We identify correlations between specific latent features and distinct domains of chemical knowledge, including structural motifs, physicochemical properties, and pharmacological drug classes. Our approach provides a generalisable framework for uncovering latent knowledge in chemistry-focused AI systems. This work has implications for both foundational understanding and practical deployment; with the potential to accelerate computational chemistry research.
Masked Autoencoders that Feel the Heart: Unveiling Simplicity Bias for ECG Analyses
State Key Laboratory of Digital Medical Engineering, Southeast University（数字医疗工程国家重点实验室，东南大学）；School of Instrument Science and Engineering, Southeast University（仪器科学与工程学院，东南大学）；School of Computer Science and Engineering, Southeast University（计算机科学与工程学院，东南大学）
Optimizing molecular design and discovering novel chemical structures to meet certain objectives, such as quantitative estimates of the drug-likeness score (QEDs), is NP-hard due to the vast combinatorial design space of discrete molecular structures, which makes it near impossible to explore the entire search space comprehensively to exploit de novo structures with properties of interest. To address this challenge, reducing the intractable search space into a lower-dimensional latent volume helps examine molecular candidates more feasibly via inverse design. Autoencoders are suitable deep lea…
Molecular Representation Learning is essential to solving many drug discovery and computational chemistry problems. It is a challenging problem due to the complex structure of molecules and the vast chemical space. Graph representations of molecules are more expressive than traditional representations, such as molecular fingerprints. Therefore, they can improve the performance of machine learning models. We propose SeMole, a method that augments the Junction Tree Variational Autoencoders, a state-of-the-art generative model for molecular graphs, with semi-supervised learning. SeMole aims to im…
The de novo design of molecular structures using deep learning generative models introduces an encouraging solution to drug discovery in the face of the continuously increased cost of new drug development. From the generation of original texts, images, and videos, to the scratching of novel molecular structures, the incredible creativity of deep learning generative models surprised us about the height machine intelligence can achieve. The purpose of this paper is to review the latest advances in generative chemistry which relies on generative modeling to expedite the drug discovery process. Th…
AstraZeneca R&D Gothenburg（阿斯利康研发哥德堡）；University of Bonn（波恩大学）
A major challenge in computational chemistry is the generation of novel molecular structures with desirable pharmacological and physiochemical properties. In this work, we investigate the potential use of autoencoder, a deep learning methodology, for de novo molecular design. Various generative autoencoders were used to map molecule structures into a continuous latent space and vice versa and their performance as structure generator was assessed. Our results show that the latent space preserves chemical similarity principle and thus can be used for the generation of analogue structures. Furthe…
Unveiling the Two-Faced Truth: Disentangling Morphed Identities for Face Morphing Detection
INESC TEC（INESC TEC（葡萄牙系统与计算机工程研究所））；University of Porto（波尔图大学）；Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形学研究所IGD）；TU Darmstadt（达姆施塔特工业大学）
Generative artificial intelligence for computational chemistry: a roadmap to predicting emergent phenomena
University of Maryland（马里兰大学）；Institute for Physical Science and Technology（物理科学与技术研究所）；University of Maryland Institute for Health Computing（马里兰大学健康计算研究所）；Department of Chemistry and Biochemistry（化学与生物化学系）；Department of Physics（物理系）；Biophysics Program（生物物理学项目）
From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models
Laboratory of Molecular Pharmacokinetics, Graduate School of Pharmaceutical Sciences, The University of Tokyo（分子药代动力学实验室，药学研究生院，东京大学）；The Institute of Statistical Mathematics (ISM), Research Organization of Information and Systems（统计数学研究所（ISM），信息与系统研究组织）
LTCI & CMAP , Télécom Paris & Ecole Polytechnique, IP Paris（巴黎电信学院与应用数学研究所、巴黎电信学院与巴黎高等理工学院、IP巴黎）；LTCI, Télécom Paris, IP Paris（巴黎电信学院、巴黎电信学院、IP巴黎）；CMAP, Ecole Polytechnique, IP Paris（应用数学研究所、巴黎高等理工学院、IP巴黎）
UT-Battelle, LLC（UT-巴特尔有限责任公司）；U.S. Department of Energy（美国能源部）
Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g., balancing the Kullback Leibler (KL) and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolut…
MIRRA: Fast, High-Fidelity Molecular Generation with Chemistry-Informed Discrete Flow Matching
Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Department of Physics, New York University（纽约大学物理系）
Drug-drug interactions (DDIs) are a major cause of preventable hospitalizations and deaths. Predicting the occurrence of DDIs helps drug safety professionals allocate investigative resources and take appropriate regulatory action promptly. Traditional DDI prediction methods predict DDIs based on the similarity between drugs. Recently, researchers revealed that predictive performance can be improved by better modeling the interactions between drug pairs with bilinear forms. However, the shallow models leveraging bilinear forms suffer from limitations on capturing complicated nonlinear interacti…
With the recent advances in machine learning for quantum chemistry, it is now possible to predict the chemical properties of compounds and to generate novel molecules. Existing generative models mostly use a string- or graph-based representation, but the precise three-dimensional coordinates of the atoms are usually not encoded. First attempts in this direction have been proposed, where autoregressive or GAN-based models generate atom coordinates. Those either lack a latent space in the autoregressive setting, such that a smooth exploration of the compound space is not possible, or cannot gene…
