The development of powerful natural language models have increased the ability to learn meaningful representations of protein sequences. In addition, advances in high-throughput mutagenesis, directed evolution, and next-generation sequencing have allowed for the accumulation of large amounts of labeled fitness data. Leveraging these two trends, we introduce Regularized Latent Space Optimization (ReLSO), a deep transformer-based autoencoder which features a highly structured latent space that is trained to jointly generate sequences as well as predict fitness. Through regularized prediction heads, ReLSO introduces a powerful protein sequence encoder and novel approach for efficient fitness landscape traversal. Using ReLSO, we explicitly model the sequence-function landscape of large labeled datasets and generate new molecules by optimizing within the latent space using gradient-based methods. We evaluate this approach on several publicly-available protein datasets, including variant sets of anti-ranibizumab and GFP. We observe a greater sequence optimization efficiency (increase in fitness per optimization step) by ReLSO compared to other approaches, where ReLSO more robustly generates high-fitness sequences. Furthermore, the attention-based relationships learned by the jointly-trained ReLSO models provides a potential avenue towards sequence-level fitness attribution information.
Determining the different conformational states of a protein and the transition paths between them is key to fully understanding the relationship between biomolecular structure and function. This can be accomplished by sampling protein conformational space with molecular simulation methodologies. Despite advances in computing hardware and sampling techniques, simulations always yield a discretized representation of this space, with transition states undersampled proportionally to their associated energy barrier. We present a convolutional neural network that learns a continuous conformational …
Massive molecular simulations of drug-target proteins have been used as a tool to understand disease mechanism and develop therapeutics. This work focuses on learning a generative neural network on a structural ensemble of a drug-target protein, e.g. SARS-CoV-2 Spike protein, obtained from computationally expensive molecular simulations. Model tasks involve characterizing the distinct structural fluctuations of the protein bound to various drug molecules, as well as efficient generation of protein conformations that can serve as an complement of a molecular simulation engine. Specifically, we …
Protein Counterfactuals via Diffusion-Guided Latent Optimization
Machine Learning Group, Technische Universität Berlin（技术大学柏林机器学习组）；Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所（BIFOLD））；BASF Digital Solutions GmbH（巴斯夫数字解决方案有限公司）
Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent Space
School of Computing, KAIST（韩国科学技术院计算机学院）；Center for Mathematical and Computational Sciences, Institute for Basic Science（基础科学研究院数学与计算科学中心）；Center for Biomolecular and Cellular Structure, Institute for Basic Science（基础科学研究院生物分子与细胞结构中心）；Department of Biological Sciences, KAIST（韩国科学技术院生物科学系）
Latent Retrieval Augmented Generation of Cross-Domain Protein Binders
Dept. of Comp. Sci. & Tech., Tsinghua University（计算机科学与技术系，清华大学）；Institute for AIR, Tsinghua University（空气智能研究所，清华大学）；Gaoling School of Artificial Intelligence, Renmin University of China（北京理工大学人工智能学院）
As deep Variational Auto-Encoder (VAE) frameworks become more widely used for modeling biomolecular simulation data, we emphasize the capability of the VAE architecture to concurrently maximize the timescale of the latent space while inferring a reduced coordinate, which assists in finding slow processes as according to the variational approach to conformational dynamics. We additionally provide evidence that the VDE framework (Hernández et al., 2017), which uses this autocorrelation loss along with a time-lagged reconstruction loss, obtains a variationally optimized latent coordinate in compa…
Generative Modeling of Full-Atom Protein Conformations using Latent Diffusion on Graph Embeddings
Signal Processing Laboratory (LTS2), EPFL, Lausanne, Switzerland（信号处理实验室（LTS2），瑞士洛桑联邦理工学院）；Institute of Bioengineering, EPFL, Lausanne, Switzerland（生物工程研究所，瑞士洛桑联邦理工学院）；Bioinformatics Group, University & Research Wageningen, The Netherlands（生物信息学组，荷兰瓦赫宁根大学及研究学院）；Ludwig Institute for Cancer Research, Lausanne, Switzerland（癌症研究路易斯研究所，瑞士洛桑）
Ophiuchus: Scalable Modeling of Protein Structures through Hierarchical Coarse-graining SO(3)-Equivariant Autoencoders
Center for Bits and Atoms（比特与原子中心）；MIT Media Lab, Molecular Machines（麻省理工学院媒体实验室，分子机器）；Atomic Architects（原子建筑师）；MIT Research Laboratory of Electronics（麻省理工学院电子研究实验室）
Korea Advanced Institute of Science and Technology（韩国科学技术院）；National University of Sciences and Technology（国立科技大学）
Understanding E3 ligase and target substrate interactions are important for cell biology and therapeutic development. However, experimental identification of E3 target relationships is not an easy task due to the labor-intensive nature of the experiments. In this article, a sequence-based E3-target prediction model is proposed for the first time. The proposed framework utilizes composition of k-spaced amino acid pairs (CKSAAP) to learn the relationship between E3 ligases and their target protein. A class separable latent space encoding scheme is also devised that provides a compressed represen…
Flow-Based Fragment Identification via Binding Site-Specific Latent Representations
Ecole Polytechnique Fédérale de Lausanne（瑞士联邦理工学院）；Oxford University（牛津大学）；VantAI；Aithyra Research Institute for Biomedical Artificial Intelligence（Aithyra生物医学人工智能研究所）；National Centre of Competence in Research (NCCR) Catalysis EPFL, Switzerland（瑞士国家研究能力中心（NCCR）催化EPFL，瑞士）
ConforNets: Latents-Based Conformational Control in OpenFold3
Department of Systems Biology, Columbia University, NY, USA（系统生物学系，哥伦比亚大学，纽约，美国）；Department of Computer Science, Columbia University, NY, USA（计算机科学系，哥伦比亚大学，纽约，美国）；Department of Computer Science, Princeton University, Princeton, NJ, USA（计算机科学系，普林斯顿大学，普林斯顿，新泽西州，美国）
Generation of drug-like molecules with high binding affinity to target proteins remains a difficult and resource-intensive task in drug discovery. Existing approaches primarily employ reinforcement learning, Markov sampling, or deep generative models guided by Gaussian processes, which can be prohibitively slow when generating molecules with high binding affinity calculated by computationally-expensive physics-based methods. We present Latent Inceptionism on Molecules (LIMO), which significantly accelerates molecule generation with an inceptionism-like technique. LIMO employs a variational aut…
OneProt: Towards Multi-Modal Protein Foundation Models
PriorLabs；Helmholtz AI（亥姆霍兹人工智能）；Technical University of Munich（慕尼黑工业大学）；Munich Data Science Institute（慕尼黑数据科学研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Institute of Bio- and Geosciences (IBG-4: Bioinformatics)（生物与地球科学研究所（IBG-4：生物信息学））；Institute for Pharmaceutical and Medicinal Chemistry（药物与医药化学研究所）；Helmholtz Munich（慕尼黑亥姆霍兹中心）
University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）
We devise an approach for targeted molecular design, a problem of interest in computational drug discovery: given a target protein site, we wish to generate a chemical with both high binding affinity to the target and satisfactory pharmacological properties. This problem is made difficult by the enormity and discreteness of the space of potential therapeutics, as well as the graph-structured nature of biomolecular surface sites. Using a dataset of protein-ligand complexes, we surmount these issues by extracting a signature of the target site with a graph convolutional network and by encoding t…
University of Cambridge（剑桥大学）；Computer Laboratory（计算机实验室）
Delays in protein synthesis cause a confounding effect when constructing Gene Regulatory Networks (GRNs) from RNA-sequencing time-series data. Accurate GRNs can be very insightful when modelling development, disease pathways, and drug side-effects. We present a model which incorporates translation delays by combining mechanistic equations and Bayesian approaches to fit to experimental data. This enables greater biological interpretability, and the use of Gaussian processes enables non-linear expressivity through kernels as well as naturally accounting for biological variation.
KTH Royal Institute of Technology（皇家理工学院）；University of Amsterdam（阿姆斯特丹大学）；University of Cambridge（剑桥大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Delft University of Technology（代尔夫特理工大学）
Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute
NVIDIA（英伟达）；University of Oxford（牛津大学）；Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；HEC Montréal（蒙特利尔高等商学院）；CIFAR AI Chair（CIFAR人工智能主席）；AITHYRA；School of Biological Sciences, Seoul National University（首尔大学生物科学学院）；Interdisciplinary Program in Bioinformatics, Seoul National University（首尔大学生物信息学跨学科项目）；Institute of Molecular Biology and Genetics, Seoul National University（首尔大学分子生物学与遗传学研究所）；Artificial Intelligence Institute, Seoul National University（首尔大学人工智能研究所）
Insights Into the Inner Workings of Transformer Models for Protein Function Prediction
Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI（弗劳恩霍夫电信研究所（海因里希·赫兹研究所））；Carl von Ossietzky University of Oldenburg（卡尔·冯·奥西茨基奥尔登堡大学）
Carnegie Mellon University（卡内基梅隆大学）；The Ohio State University（俄亥俄州立大学）
Despite the prevalence and many successes of deep learning applications in de novo molecular design, the problem of peptide generation targeting specific proteins remains unsolved. A main barrier for this is the scarcity of the high-quality training data. To tackle the issue, we propose a novel machine learning based peptide design architecture, called Latent Space Approximate Trajectory Collector (LSATC). It consists of a series of samplers on an optimization trajectory on a highly non-convex energy landscape that approximates the distributions of peptides with desired properties in a latent …
Attribute extrapolation in sample generation is challenging for deep neural networks operating beyond the training distribution. We formulate a new task for extrapolation in sequence generation, focusing on natural language and proteins, and propose GENhance, a generative framework that enhances attributes through a learned latent space. Trained on movie reviews and a computed protein stability dataset, GENhance can generate strongly-positive text reviews and highly stable protein sequences without being exposed to similar data during training. We release our benchmark tasks and models to cont…
Structure Language Models for Protein Conformation Generation
Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；University of Ottawa（渥太华大学）；National Research Council Canada（加拿大国家研究委员会）；CIFAR AI Chair（加拿大高级研究所人工智能主席项目）；HEC Montréal（蒙特利尔高等商学院）
