Generative modeling for protein engineering is key to solving fundamental problems in synthetic biology, medicine, and material science. We pose protein engineering as an unsupervised sequence generation problem in order to leverage the exponentially growing set of proteins that lack costly, structural annotations. We train a 1.2B-parameter language model, ProGen, on ~280M protein sequences conditioned on taxonomic and keyword tags such as molecular function and cellular component. This provides ProGen with an unprecedented range of evolutionary sequence diversity and allows it to generate with fine-grained control as demonstrated by metrics based on primary sequence similarity, secondary structure accuracy, and conformational energy.
Deep learning is catalyzing a scientific revolution fueled by big data, accessible toolkits, and powerful computational resources, impacting many fields including protein structural modeling. Protein structural modeling, such as predicting structure from amino acid sequence and evolutionary information, designing proteins toward desirable functionality, or predicting properties or behavior of a protein, is critical to understand and engineer biological systems at the molecular level. In this review, we summarize the recent advances in applying deep learning techniques to tackle problems in pro…
Expanding the scope of graph-based, deep-learning models to noncovalent protein-ligand interactions has earned increasing attention in structure-based drug design. Modeling the protein-ligand interactions with graph neural networks (GNNs) has experienced difficulties in the conversion of protein-ligand complex structures into the graph representation and left questions regarding whether the trained models properly learn the appropriate noncovalent interactions. Here, we proposed a GNN architecture, denoted as InteractionNet, which learns two separated molecular graphs, being covalent and nonco…
Massachusetts Institute of Technology（麻省理工学院）；University of Washington（华盛顿大学）
Construction of a scaffold structure that supports a desired motif, conferring protein function, shows promise for the design of vaccines and enzymes. But a general solution to this motif-scaffolding problem remains open. Current machine-learning techniques for scaffold design are either limited to unrealistically small scaffolds (up to length 20) or struggle to produce multiple diverse scaffolds. We propose to learn a distribution over diverse and longer protein backbone structures via an E(3)-equivariant graph neural network. We develop SMCDiff to efficiently sample scaffolds from this distr…
Tsinghua University（清华大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；Tencent Quantum Lab（腾讯量子实验室）
Protein is linked to almost every life process. Therefore, analyzing the biological structure and property of protein sequences is critical to the exploration of life, as well as disease detection and drug discovery. Traditional protein analysis methods tend to be labor-intensive and time-consuming. The emergence of deep learning models makes modeling data patterns in large quantities of data possible. Interdisciplinary researchers have begun to leverage deep learning methods to model large biological datasets, e.g. using long short-term memory and convolutional neural network for protein sequ…
University of Illinois at Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）；Tsinghua University（清华大学）
Modeling the effects of mutations on the binding affinity plays a crucial role in protein engineering and drug design. In this study, we develop a novel deep learning based framework, named GraphPPI, to predict the binding affinity changes upon mutations based on the features provided by a graph neural network (GNN). In particular, GraphPPI first employs a well-designed pre-training scheme to enforce the GNN to capture the features that are predictive of the effects of mutations on binding affinity in an unsupervised manner and then integrates these graphical features with gradient-boosting tr…
Protein modeling is an increasingly popular area of machine learning research. Semi-supervised learning has emerged as an important paradigm in protein modeling due to the high cost of acquiring supervised protein labels, but the current literature is fragmented when it comes to datasets and standardized evaluation techniques. To facilitate progress in this field, we introduce the Tasks Assessing Protein Embeddings (TAPE), a set of five biologically relevant semi-supervised learning tasks spread across different domains of protein biology. We curate tasks into specific training, validation, an…
ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling
Department of Computer Science, Tsinghua University（清华大学计算机系）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；PharMolix Inc（PharMolix公司）
University of Washington（华盛顿大学）；Salesforce Research（赛富时研究院）
For protein sequence datasets, unlabeled data has greatly outpaced labeled data due to the high cost of wet-lab characterization. Recent deep-learning approaches to protein prediction have shown that pre-training on unlabeled data can yield useful representations for downstream tasks. However, the optimal pre-training strategy remains an open question. Instead of strictly borrowing from natural language processing (NLP) in the form of masked or autoregressive language modeling, we introduce a new pre-training task: directly predicting protein profiles derived from multiple sequence alignments.…
MutaPLM: Protein Language Modeling for Mutation Explanation and Engineering
Institute of AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Pharmolix Inc.（Pharmolix公司）
Seoul National University（首尔国立大学）；LG AI Research（LG人工智能研究院）；NAVER Corp.（NAVER公司）；Kangwon National University（江原国立大学）；Seoul National University of Science and Technology（首尔科技大学）
Bridging the exponentially growing gap between the numbers of unlabeled and labeled protein sequences, several studies adopted semi-supervised learning for protein sequence modeling. In these studies, models were pre-trained with a substantial amount of unlabeled data, and the representations were transferred to various downstream tasks. Most pre-training methods solely rely on language modeling and often exhibit limited performance. In this paper, we introduce a novel pre-training scheme called PLUS, which stands for Protein sequence representations Learned Using Structural information. PLUS …
Protein structure prediction remains to be an open problem in bioinformatics. There are two main categories of methods for protein structure prediction: Free Modeling (FM) and Template Based Modeling (TBM). Protein threading, belonging to the category of template based modeling, identifies the most likely fold with the target by making a sequence-structure alignment between target protein and template protein. Though protein threading has been shown to more be successful for protein structure prediction, it performs poorly for remote homology detection.
Renmin University of China（中国人民大学）；DP Technology（深势科技）
The prediction of protein-ligand binding affinity is of great significance for discovering lead compounds in drug research. Facing this challenging task, most existing prediction methods rely on the topological and/or spatial structure of molecules and the local interactions while ignoring the multi-level inter-molecular interactions between proteins and ligands, which often lead to sub-optimal performance. To solve this issue, we propose a novel global-local interaction (GLI) framework to predict protein-ligand binding affinity. In particular, our GLI framework considers the inter-molecular i…
We investigate the measurement of DNA supercoiling density ($σ$) along chromosomes using interaction frequencies between DNA and DNA-anchored clusters of proteins. Specifically, we show how the physics of DNA supercoiling leads, in bacteria, to the quantitative modeling of binding properties of ParB proteins around their centromere-like site, {\it parS}. Using this framework, we provide an upper bound for $σ$ in the {\it Escherichia coli} chromosome, consistent with plasmid values, and offer a proof of concept for a high accuracy measurement. To reach these conclusions, we revisit the problem …
Despite the significant increase in computational power, molecular modeling of protein structure using classical all-atom approaches remains inefficient, at least for most of the protein targets in the focus of biomedical research. Perhaps the most successful strategy to overcome the inefficiency problem is multiscale modeling to merge all-atom and coarse-grained models. This chapter describes a well-established CABS coarse-grained protein model. The CABS (C-Alpha, C-Beta and Side chains) model assumes a 2-4 united-atom representation of amino acids, knowledge-based force field (derived from t…
Proteins are an important class of biomolecules that serve as essential building blocks of the cells. Their three-dimensional structures are responsible for their functions. In this thesis we have investigated the protein structures using a network theoretical approach. While doing so we used a coarse-grained method, viz., complex network analysis. We model protein structures at two length scales as Protein Contact Networks (PCN) and as Long-range Interaction Networks (LINs). We found that proteins by virtue of being characterised by high amount of clustering, are small-world networks. Apart f…
Small single-domain proteins often exhibit only a single free-energy barrier, or transition state, between the denatured and the native state. The folding kinetics of these proteins is usually explored via mutational analysis. A central question is which structural information on the transition state can be derived from the mutational data. In this article, we model and structurally interpret mutational Phi-values for two small beta-sheet proteins, the PIN and the FBP WW domain. The native structure of these WW domains comprises two beta-hairpins that form a three-stranded beta-sheet. In our m…
Computational methods are needed to differentiate the small fraction of missense mutations that contribute to disease by disrupting protein function from neutral variants. We describe several complementary methods using large-scale homology modeling of human protein complexes to detect non-neutral mutations. Importantly, unlike sequence conservation-based methods, this structure-based approach provides experimentally testable biochemical mechanisms for mutations in disease. Specifically, we infer metal ion, small molecule, protein-protein, and nucleic acid binding sites by homology and find th…
Massachusetts Institute of Technology（麻省理工学院）；Simons Machine Learning Center（西蒙斯机器学习中心）；New York Structural Biology Center（纽约结构生物学中心）
We present the MSA-to-protein transformer, a generative model of protein sequences conditioned on protein families represented by multiple sequence alignments (MSAs). Unlike existing approaches to learning generative models of protein families, the MSA-to-protein transformer conditions sequence generation directly on a learned encoding of the multiple sequence alignment, circumventing the need for fitting dedicated family models. By training on a large set of well-curated multiple sequence alignments in Pfam, our MSA-to-protein transformer generalizes well to protein families not observed duri…
Proteins are polymers of amino acids. These macromolecules are synthesized by intracellular machines called ribosomes. Although the experimental investigation of protein synthesis has been a traditional area of research in molecular cell biology, important quantitative models of protein synthesis have been reported in research journals devoted to statistical physics and related interdisciplinary topics. From the perspective of a physicist, protein synthesis is the classical transport of interacting ribosomes on a messenger RNA (mRNA) template that dictates the sequence of the amino acids on th…
Protein-peptide interactions play essential functional roles in living organisms and their structural characterization is a hot subject of current experimental and theoretical research. Computational modeling of the structure of protein-peptide interactions is usually divided into two stages: prediction of the binding site at a protein receptor surface, and then docking (and modeling) the peptide structure into the known binding site. This paper presents a comprehensive CABS-dock method for the simultaneous search of binding sites and flexible protein-peptide docking, available as a users frie…
Multi-level Interaction Modeling for Protein Mutational Effect Prediction
School of Information and Software Engineering, UESTC（电子科技大学信息与软件工程学院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；School of Life Sciences, Tsinghua University（清华大学生命科学学院）
Inside a cell, heterotypic proteins assemble in inhomogeneous, crowded systems where the abundance of these proteins vary with cell types. While some protein complexes form putative structures that can be visualized with imaging, there are far more protein complexes that are yet to be solved because of their dynamic associations with one another. Yet, it is possible to infer these protein complexes through a physical model. However, it is often not clear to physicists what kind of data from biology is necessary for such a modeling endeavor. Here, we aim to model these clusters of coarse-graine…
Although rare, an increasing number of proteins have been observed to contain entanglements in their native structures. To gain more insight into the significance of protein knotting, researchers have been investigating protein knot formation using both experimental and theoretical methods. Motivated by the hypothesized folding pathway of $α$-haloacid dehalogenase (DehI) protein, Flapan, He, and Wong proposed a theory of how protein knots form, which includes existing folding pathways described by Taylor and Bölinger et al. as special cases. In their topological descriptions, two loops in an u…
Motivation: Due to the central role of protein structure in molecular recognition, great computational efforts are devoted to modeling protein structures and motions that mediate structural rearrangements. The size, dimensionality, and non-linearity of the protein structure space present outstanding challenges. Such challenges also arise in robot motion planning, and robotics-inspired treatments of protein structure and motion are increasingly showing high exploration capability. Encouraged by such findings, we debut here ROMEO, which stands for Robotics prOtein Motion ExplOration framework. R…
Motivation Protein fold recognition is an important problem in structural bioinformatics. Almost all traditional fold recognition methods use sequence (homology) comparison to indirectly predict the fold of a tar get protein based on the fold of a template protein with known structure, which cannot explain the relationship between sequence and fold. Only a few methods had been developed to classify protein sequences into a small number of folds due to methodological limitations, which are not generally useful in practice. Results We develop a deep 1D-convolution neural network (DeepSF) to dire…
During their evolution, proteins explore sequence space via an interplay between random mutations and phenotypic selection. Here we build upon recent progress in reconstructing data-driven fitness landscapes for families of homologous proteins, to propose stochastic models of experimental protein evolution. These models predict quantitatively important features of experimentally evolved sequence libraries, like fitness distributions and position-specific mutational spectra. They also allow us to efficiently simulate sequence libraries for a vast array of combinations of experimental parameters…
We present a dual optimization concept of predicting optimal sequences as well as optimal folds of off-lattice protein models in the context of multi-scale modeling. We validate the utility of the recently introduced hidden-force Monte Carlo optimization algorithm by finding significantly lower energy folds for minimalist and detailed protein models than previously reported. Further, we also find the protein sequence that yields the lowest energy fold amongst all sequences for a given chain length and residue mixture. In particular, for protein models with a binary sequence, we show that the s…
A statistical model of protein families, called profile conditional random fields (CRFs), is proposed. This model may be regarded as an integration of the profile hidden Markov model (HMM) and the Finkelstein-Reva (FR) theory of protein folding. While the model structure of the profile CRF is almost identical to the profile HMM, it can incorporate arbitrary correlations in the sequences to be aligned to the model. In addition, like in the FR theory, the profile CRF can incorporate long-range pairwise interactions between model states via mean-field-like approximations. We give the detailed for…
Protriever: End-to-End Differentiable Protein Homology Search for Fitness Prediction
Department of Systems Biology, Harvard Medical School（系统生物学系，哈佛医学院）；Department of Computer Science, University of Oxford（计算机科学系，牛津大学）；Department of Computer Science, University of Copenhagen（计算机科学系，哥本哈根大学）；Enzyme research, Novonesis（酶研究，Novonesis）
Generative Modeling of Full-Atom Protein Conformations using Latent Diffusion on Graph Embeddings
Signal Processing Laboratory (LTS2), EPFL, Lausanne, Switzerland（信号处理实验室（LTS2），瑞士洛桑联邦理工学院）；Institute of Bioengineering, EPFL, Lausanne, Switzerland（生物工程研究所，瑞士洛桑联邦理工学院）；Bioinformatics Group, University & Research Wageningen, The Netherlands（生物信息学组，荷兰瓦赫宁根大学及研究学院）；Ludwig Institute for Cancer Research, Lausanne, Switzerland（癌症研究路易斯研究所，瑞士洛桑）
We investigate protein-protein interactions in solution by small-angle X-ray scattering (SAXS) and theoretical modeling. The structure factor for solutions of bovine pancreatic trypsin inhibitor (BPTI), myoglobin (Mb), and intestinal fatty acid-binding protein (IFABP) is determined from SAXS measurements at multiple concentrations, from Monte Carlo simulations with a coarse-grained structure-based interaction model, and from analytic approximate solutions of two idealized colloidal interaction models without adjustable parameters. By combining these approaches, we find that the structure facto…
Ryerson University（瑞尔森大学）；Istituto Superiore di Sanità（意大利国家高等卫生研究院）
This paper deals with the relations among structural, topological, and chemical properties of the E.Coli proteome from the vantage point of the solubility/aggregation propensity of proteins. Each E.Coli protein is initially represented according to its known folded 3D shape. This step consists in representing the available E.Coli proteins in terms of graphs. We first analyze those graphs by considering pure topological characterizations, i.e., by analyzing the mass fractal dimension and the distribution underlying both shortest paths and vertex degrees. Results confirm the general architectura…
Proteins play crucial roles in every cellular process by interacting with each other, with nucleic acids, metabolites, and other molecules. The resulting assemblies can be very large and intricate and pose challenges to experimental methods. In the current era of integrative modeling, it is often only by a combination of various experimental techniques and computations that 3D models of those molecular machines can be obtained. Among the various computational approaches available, molecular docking is often the method of choice when it comes to predicting 3D structures of complexes. Docking ca…
