Boosting Convolutional Neural Networks' Protein Binding Site Prediction Capacity Using SE(3)-invariant transformers, Transfer Learning and Homology-based Augmentation
Daeseok Lee, Jeunghyun Byun, Bonggun Shin
备注：
Updates in version 2: author order change (making it clear that Bonggun Shin is the corresponding author)
Figuring out small molecule binding sites in target proteins, in the resolution of either pocket or residue, is critical in many virtual and real drug-discovery scenarios. Since it is not always easy to find such binding sites based on domain knowledge or traditional methods, different deep learning methods that predict binding sites out of protein structures have been developed in recent years. Here we present a new such deep learning algorithm, that significantly outperformed all state-of-the-art baselines in terms of the both resolutions$\unicode{x2013}$pocket and residue. This good performance was also demonstrated in a case study involving the protein human serum albumin and its binding sites. Our algorithm included new ideas both in the model architecture and in the training method. For the model architecture, it incorporated SE(3)-invariant geometric self-attention layers that operate on top of residue-level CNN outputs. This residue-level processing of the model allowed a transfer learning between the two resolutions, which turned out to significantly improve the binding pocket prediction. Moreover, we developed novel augmentation method based on protein homology, which prevented our model from over-fitting. Overall, we believe that our contribution to the literature is twofold. First, we provided a new computational method for binding site prediction that is relevant to real-world applications, as shown by the good performance on different benchmarks and case study. Second, the novel ideas in our method$\unicode{x2013}$the model architecture, transfer learning and the homology augmentation$\unicode{x2013}$would serve as useful components in future works.
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；University of Chicago（芝加哥大学）；University of Hong Kong（香港大学）
Computational prediction of membrane protein (MP) structures is very challenging partially due to lack of sufficient solved structures for homology modeling. Recently direct evolutionary coupling analysis (DCA) sheds some light on protein contact prediction and accordingly, contact-assisted folding, but DCA is effective only on some very large-sized families since it uses information only in a single protein family. This paper presents a deep transfer learning method that can significantly improve MP contact prediction by learning contact patterns and complex sequence-contact relationship from…
Structure determination is key to understanding protein function at a molecular level. Whilst significant advances have been made in predicting structure and function from amino acid sequence, researchers must still rely on expensive, time-consuming analytical methods to visualise detailed protein conformation. In this study, we demonstrate that it is possible to make accurate ($\geq$80%) predictions of protein class and architecture from structures determined at low ($>$3A) resolution, using a deep convolutional neural network trained on high-resolution ($\leq$3A) structures represented as 2D…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；University of Chicago（芝加哥大学）；University of Hong Kong（香港大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Computational elucidation of membrane protein (MP) structures is challenging partially due to lack of sufficient solved structures for homology modeling. Here we describe a high-throughput deep transfer learning method that first predicts MP contacts by learning from non-membrane proteins (non-MPs) and then predicting three-dimensional structure models using the predicted contacts as distance restraints. Tested on 510 non-redundant MPs, our method has contact prediction accuracy at least 0.18 better than existing methods, predicts correct folds for 218 MPs (TMscore at least 0.6), and generates…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
Sequence-based protein homology detection has been extensively studied and so far the most sensitive method is based upon comparison of protein sequence profiles, which are derived from multiple sequence alignment (MSA) of sequence homologs in a protein family. A sequence profile is usually represented as a position-specific scoring matrix (PSSM) or an HMM (Hidden Markov Model) and accordingly PSSM-PSSM or HMM-HMM comparison is used for homolog detection. This paper presents a new homology detection method MRFalign, consisting of three key components: 1) a Markov Random Fields (MRF) representa…
Given the amino acid sequence of a protein, researchers often infer its structure and function by finding homologous, or evolutionarily-related, proteins of known structure and function. Since structure is typically more conserved than sequence over long evolutionary distances, recognizing remote protein homologs from their sequence poses a challenge. We first consider all proteins of known three-dimensional structure, and explore how they cluster according to different levels of homology. An automatic computational method reasonably approximates a human-curated hierarchical organization of pr…
Protriever: End-to-End Differentiable Protein Homology Search for Fitness Prediction
Department of Systems Biology, Harvard Medical School（系统生物学系，哈佛医学院）；Department of Computer Science, University of Oxford（计算机科学系，牛津大学）；Department of Computer Science, University of Copenhagen（计算机科学系，哥本哈根大学）；Enzyme research, Novonesis（酶研究，Novonesis）
The similarity between protein sequences is a directly and easly computed quantity from which to deduce information about their evolutionary distance and to detect homologous proteins. The SIMAP database -- Similarity Matrix of Proteins -- provides a pre-computed similarity matrix covering the similarity space formed by about all publicly available amino acid sequences from public databases and completely sequenced genomes. From SIMAP we construct the protein homology network, where the proteins are the nodes and the links represent homology relationships. With more than 5 million nodes and ab…
Recently, persistent homology has had tremendous success in biomolecular data analysis. It works by examining the topological relationship or connectivity of a group of atoms in a molecule at a variety of scales, then rendering a family of topological representations of the molecule. However, persistent homology is rarely {employed} for the analysis of atomic properties, such as biomolecular flexibility analysis or B factor prediction. This work introduces atom-specific persistent homology to provide a local atomic level representation of a molecule via a global topological tool. This is achie…
Motivation: Site directed mutagenesis is widely used to understand the structure and function of biomolecules. Computational prediction of protein mutation impacts offers a fast, economical and potentially accurate alternative to laboratory mutagenesis. Most existing methods rely on geometric descriptions, this work introduces a topology based approach to provide an entirely new representation of protein mutation impacts that could not be obtained from conventional techniques. Results: Topology based mutation predictor (T-MP) is introduced to dramatically reduce the geometric complexity and nu…
Protein-ligand binding is a fundamental biological process that is paramount to many other biological processes, such as signal transduction, metabolic pathways, enzyme construction, cell secretion, gene expression, etc. Accurate prediction of protein-ligand binding affinities is vital to rational drug design and the understanding of protein-ligand binding and binding induced function. Existing binding affinity prediction methods are inundated with geometric detail and involve excessively high dimensions, which undermines their predictive power for massive binding data. Topology provides an ul…
Protein structure prediction remains to be an open problem in bioinformatics. There are two main categories of methods for protein structure prediction: Free Modeling (FM) and Template Based Modeling (TBM). Protein threading, belonging to the category of template based modeling, identifies the most likely fold with the target by making a sequence-structure alignment between target protein and template protein. Though protein threading has been shown to more be successful for protein structure prediction, it performs poorly for remote homology detection.
Protein Fragment Motif Finder (PFMFind) is a system that enables efficient discovery of relationships between short fragments of protein sequences using similarity search. It supports queries based on score matrices and PSSMs obtained through an iterative procedure similar to PSI-BLAST. PSSM construction is customisable through plugins written in Python. PFMFind consists of a GUI client, an algorithm using an index for fast similarity search and a relational database for storing search results and sequence annotations. It is written mostly in Python. All components communicate between themselv…
University of Illinois at Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）；Tsinghua University（清华大学）
Modeling the effects of mutations on the binding affinity plays a crucial role in protein engineering and drug design. In this study, we develop a novel deep learning based framework, named GraphPPI, to predict the binding affinity changes upon mutations based on the features provided by a graph neural network (GNN). In particular, GraphPPI first employs a well-designed pre-training scheme to enforce the GNN to capture the features that are predictive of the effects of mutations on binding affinity in an unsupervised manner and then integrates these graphical features with gradient-boosting tr…
Graph neural networks are a quickly emerging field for non-Euclidean data that leverage the inherent graphical structure to predict node, edge, and global-level properties of a system. Protein properties can not easily be understood as a simple sum of their parts (i.e. amino acids), therefore, understanding their dynamical properties in the context of graphs is attractive for revealing how perturbations to their structure can affect their global function. To tackle this problem, we generate a database of 2020 mutated calponin homology (CH) domains undergoing large-scale separation in molecular…
Improving homology-directed repair by small molecule agents for genetic engineering in unconventional yeast? -- Learning from the engineering of mammalian systems
Generative power of a protein language model trained on multiple sequence alignments
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Institute of Bioengineering（生物工程研究所）；School of Life Sciences（生命科学学院）；SIB Swiss Institute of Bioinformatics（瑞士生物信息学研究所）
High throughput sequencing techniques have highly impactedon modern biology, widening the gap between sequenced andannotated data. Automatic annotation tools are thereforeof the foremost importance to guide biologists' experiments. However, most of the state-of-the-art methods rely on annotation transfer, offering reliable predictions only in homology settings. In this work we present a novel appraoch to protein feature prediction, which exploits the Semanti Based Regularization to inject prior knowledge in the learning process. The experimental results conducted on the yeast genome show that …
Identification of high affinity drug-target interactions is a major research question in drug discovery. Proteins are generally represented by their structures or sequences. However, structures are available only for a small subset of biomolecules and sequence similarity is not always correlated with functional similarity. We propose ChemBoost, a chemical language based approach for affinity prediction using SMILES syntax. We hypothesize that SMILES is a codified language and ligands are documents composed of chemical words. These documents can be used to learn chemical word vectors that repre…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
Recently exciting progress has been made on protein contact prediction, but the predicted contacts for proteins without many sequence homologs is still of low quality and not very useful for de novo structure prediction. This paper presents a new deep learning method that predicts contacts by integrating both evolutionary coupling (EC) and sequence conservation information through an ultra-deep neural network formed by two deep residual networks. This deep neural network allows us to model very complex sequence-contact relationship as well as long-range inter-contact correlation. Our method gr…
It has long been observed that trimethylamin N-oxide (TMAO) and urea demonstrate dramatically different properties in a protein folding process. Even with the enormous theoretical and experimental research work of the two osmolytes, various aspects of their underlying mechanisms still remain largely elusive. In this paper, we propose to use the weighted persistent homology to systematically study the osmolytes molecular aggregation and their hydrogen-bonding network from a local topological perspective. We consider two weighted models, i.e., localized persistent homology (LPH) and interactive …
Although deep learning approaches have had tremendous success in image, video and audio processing, computer vision, and speech recognition, their applications to three-dimensional (3D) biomolecular structural data sets have been hindered by the entangled geometric complexity and biological complexity. We introduce topology, i.e., element specific persistent homology (ESPH), to untangle geometric complexity and biological complexity. ESPH represents 3D complex geometry by one-dimensional (1D) topological invariants and retains crucial biological information via a multichannel image representat…
RegimeFormer: A Large Protein Model of Global Perturbation Regimes
Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）；Alibaba-NTU Global e-Sustainability CorpLab (ANGEL)（阿里巴巴-南洋理工大学全球电子可持续发展联合实验室）；Shanghai Jiao Tong University（上海交通大学）；Zhejiang A&F University（浙江农林大学）；University of Oxford（牛津大学）；Shanghai East Hospital, Tongji University School of Medicine（同济大学医学院附属东方医院）；University of Southern California（南加州大学）；Zhejiang University（浙江大学）；Jinan University（暨南大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
The devastating impact of the COVID-19 pandemic caused by SARS coronavirus 2 (SARS CoV 2) has raised important questions about viral origin, mechanisms of zoonotic transfer to humans, whether companion or commercial animals can act as reservoirs for infection, and why there are large variations in SARS-CoV-2 susceptibilities across animal species. Powerful in silico modelling methods can rapidly generate information on newly emerged pathogens to aid countermeasure development and predict future behaviours. Here we report an in silico structural homology modelling, protein-protein docking, and …
Two types of osmolytes, i.e., trimethylamin N-oxide (TMAO) and urea, demonstrate dramatically different properties in a protein folding process. Even with the great progresses in revealing the potential underlying mechanism of these two osmolyte systems, many problems still remain unsolved. In this paper, we propose to use the persistent homology, a newly-invented topological method, to systematically study the osmolytes molecular aggregation and their hydrogen-bonding network from a global topological perspective. It has been found that, for the first time, TMAO and urea show two extremely di…
