Machine learning and the use of neural networks has increased precipitously over the past few years primarily due to the ever-increasing accessibility to data and the growth of computation power. It has become increasingly easy to harness the power of machine learning for predictive tasks. Protein structure prediction is one area where neural networks are becoming increasingly popular and successful. Although very powerful, the use of ANN require selection of most appropriate input/output encoding, architecture, and class to produce the optimal results. In this investigation we have explored and evaluated the effect of several conventional and newly proposed input encodings and selected an optimal architecture. We considered 11 variations of input encoding, 11 alternative window sizes, and 7 different architectures. In total, we evaluated 2,541 permutations in application to the training and testing of more than 10,000 protein structures over the course of 3 months. Our investigations concluded that one-hot encoding, the use of LSTMs, and window sizes of 9, 11, and 15 produce the optimal outcome. Through this optimization, we were able to improve the quality of protein structure prediction by predicting the ϕ dihedrals to within 14° - 16° and ψ dihedrals to within 23°- 25°. This is a notable improvement compared to previously similar investigations.
Harvard University（哈佛大学）；Harvard Medical School（哈佛医学院）
Proteins are responsible for the most diverse set of functions in biology. The ability to extract information from protein sequences and to predict the effects of mutations is extremely valuable in many domains of biology and medicine. However the mapping between protein sequence and function is complex and poorly understood. Here we present an embedding of natural protein sequences using a Variational Auto-Encoder and use it to predict how mutations affect protein function. We use this unsupervised approach to cluster natural variants and learn interactions between sets of positions within a …
Shiraz University of Medical Science（设拉子医科大学）；University of Tehran（德黑兰大学）
Protein-Protein Interactions (PPIs) perform essential roles in biological functions. Although some experimental techniques have been developed to detect PPIs, they suffer from high false positive and high false negative rates. Consequently, efforts have been devoted during recent years to develop computational approaches to predict the interactions utilizing various sources of information. Therefore, a unique category of prediction approaches has been devised which is based on the protein sequence information. However, finding an appropriate feature encoding to characterize the sequence of pro…
Recent advances in protein function prediction exploit graph-based deep learning approaches to correlate the structural and topological features of proteins with their molecular functions. However, proteins in vivo are not static but dynamic molecules that alter conformation for functional purposes. Here we apply normal mode analysis to native protein conformations and augment protein graphs by connecting edges between dynamically correlated residue pairs. In the multilabel function classification task, our method demonstrates a remarkable performance gain based on this dynamics-informed repre…
As deep Variational Auto-Encoder (VAE) frameworks become more widely used for modeling biomolecular simulation data, we emphasize the capability of the VAE architecture to concurrently maximize the timescale of the latent space while inferring a reduced coordinate, which assists in finding slow processes as according to the variational approach to conformational dynamics. We additionally provide evidence that the VDE framework (Hernández et al., 2017), which uses this autocorrelation loss along with a time-lagged reconstruction loss, obtains a variationally optimized latent coordinate in compa…
Golisano College of Computing and Information Sciences, Rochester Institute of Technology（罗切斯特理工学院戈利萨诺计算与信息科学学院）；Thomas H. Gosnell School of Life Sciences, Rochester Institute of Technology（罗切斯特理工学院托马斯·H·戈斯内尔生命科学学院）
Predicting protein-protein interactions (PPIs) by learning informative representations from amino acid sequences is a challenging yet important problem in biology. Although various deep learning models in Siamese architecture have been proposed to model PPIs from sequences, these methods are computationally expensive for a large number of PPIs due to the pairwise encoding process. Furthermore, these methods are difficult to interpret because of non-intuitive mappings from protein sequences to their sequence representation. To address these challenges, we present a novel deep framework to model…
Korea Advanced Institute of Science and Technology（韩国科学技术院）；National University of Sciences and Technology（国立科技大学）
Understanding E3 ligase and target substrate interactions are important for cell biology and therapeutic development. However, experimental identification of E3 target relationships is not an easy task due to the labor-intensive nature of the experiments. In this article, a sequence-based E3-target prediction model is proposed for the first time. The proposed framework utilizes composition of k-spaced amino acid pairs (CKSAAP) to learn the relationship between E3 ligases and their target protein. A class separable latent space encoding scheme is also devised that provides a compressed represen…
The University of Hong Kong（香港大学）；Xiaohe Healthcare, ByteDance（字节跳动小荷健康）；JancsiTech（Jancsi科技）；OPPO HealthLab（OPPO健康实验室）
Protein representation learning has primarily benefited from the remarkable development of language models (LMs). Accordingly, pre-trained protein models also suffer from a problem in LMs: a lack of factual knowledge. The recent solution models the relationships between protein and associated knowledge terms as the knowledge encoding objective. However, it fails to explore the relationships at a more granular level, i.e., the token level. To mitigate this, we propose Knowledge-exploited Auto-encoder for Protein (KeAP), which performs token-level knowledge graph exploration for protein represen…
GLProtein: Global-and-Local Structure Aware Protein Representation Learning
Department of Computing (COMP), The Hong Kong Polytechnic University（计算机学院（COMP），香港理工大学）；Department of Management and Marketing (MM), The Hong Kong Polytechnic University（管理与营销学院（MM），香港理工大学）；Department of Computer Science, Sichuan University（计算机学院，四川大学）
Large self-supervised models pretrained on millions of protein sequences have recently gained popularity in generating embeddings of protein sequences for protein function prediction. However, the absence of random baselines makes it difficult to conclude whether pretraining has learned useful information for protein function prediction. Here we show that one-hot encoding and random embeddings, both of which do not require any pretraining, are strong baselines for protein function prediction across 14 diverse sequence-to-function tasks.
ProteoKnight: Convolution-based Phage Virion Protein Classification and Uncertainty Analysis
Computer Science and Engineering（计算机科学与工程）；BRAC University（布拉克大学）；Computing for Sustainability and Social Good (C2SG) Research Group（可持续性与社会公益（C2SG）研究组）
General Multimodal Protein Design Enables DNA-Encoding of Chemistry
California Institute of Technology（加州理工学院）；Mila – Québec AI Institute（Mila – 魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Université Paris-Saclay（巴黎-萨克雷大学）；McGill University（麦吉尔大学）；University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；Imperial College London（伦敦帝国理工学院）；Institut Courtois（库尔图瓦研究所）；LawZero；AITHYRA；FutureHouse
Inferred global dense residue transition graphs from primary structure sequences enable protein interaction prediction via directed graph convolutional neural networks
Division of Computer Science Texas Woman’s University（计算机科学系德克萨斯女子大学）；Department of Computer Science The University of Texas Rio Grande Valley（计算机科学系德克萨斯大学里奥格兰德谷分校）
In this paper we experiment with using neural network structures to predict a protein's secondary structure (α helix positions) from only its primary structure (amino acid sequence). We implement a fully connected neural network (FCNN) and preform three experiments using that FCNN. Firstly, we do a cross-species comparison of models trained and tested on mouse and human datasets. Secondly, we test the impact of varying the length of protein sequence we input into the model. Thirdly, we compare custom error functions designed to focus on the center of the input window. At the end of paper we pr…
University of Puerto Rico（波多黎各大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
Capsule Networks have great potential to tackle problems in structural biology because of their attention to hierarchical relationships. This paper describes the implementation and application of a Capsule Network architecture to the classification of RAS protein family structures on GPU-based computational resources. The proposed Capsule Network trained on 2D and 3D structural encodings can successfully classify HRAS and KRAS structures. The Capsule Network can also classify a protein-based dataset derived from a PSI-BLAST search on sequences of KRAS and HRAS mutations. Our results show an ac…
Meta-Learning for Cross-Task Generalization in Protein Mutation Property Prediction
Department of Chemical Engineering, Carnegie Mellon University, PA 15213, USA（化学工程系，卡内基梅隆大学）；Department of Electrical and Computer Engineering, Carnegie Mellon University, PA 15213, USA（电气与计算机工程系，卡内基梅隆大学）；Department of Chemical and Biomolecular Engineering, University of Nebraska--Lincoln, NE 68588, USA（化学与生物分子工程系，内布拉斯加大学林肯分校）；Department of Biomedical Engineering, Carnegie Mellon University, PA 15213, USA（生物医学工程系，卡内基梅隆大学）；Department of Mechanical Engineering, Carnegie Mellon University, PA 15213, USA（机械工程系，卡内基梅隆大学）
Wyss Institute for Biologically Inspired Engineering（威斯生物启发工程研究所）；School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院）；Department of Systems Biology, Harvard Medical School（哈佛医学院系统生物学系）
As high-throughput biological sequencing becomes faster and cheaper, the need to extract useful information from sequencing becomes ever more paramount, often limited by low-throughput experimental characterizations. For proteins, accurate prediction of their functions directly from their primary amino-acid sequences has been a long standing challenge. Here, machine learning using artificial recurrent neural networks (RNN) was applied towards classification of protein function directly from primary sequence without sequence alignment, heuristic scoring or feature engineering. The RNN models co…
Generative Adversarial Networks (GANs) represent an attractive and novel approach to generate realistic data, such as genes, proteins, or drugs, in synthetic biology. Here, we apply GANs to generate synthetic DNA sequences encoding for proteins of variable length. We propose a novel feedback-loop architecture, called Feedback GAN (FBGAN), to optimize the synthetic gene sequences for desired properties using an external function analyzer. The proposed architecture also has the advantage that the analyzer need not be differentiable. We apply the feedback-loop mechanism to two examples: 1) genera…
The characterization of drug-protein interactions is crucial in the high-throughput screening for drug discovery. The deep learning-based approaches have attracted attention because they can predict drug-protein interactions without trial-and-error by humans. However, because data labeling requires significant resources, the available protein data size is relatively small, which consequently decreases model performance. Here we propose two methods to construct a deep learning framework that exhibits superior performance with a small labeled dataset. At first, we use transfer learning in encodi…
Attending on Multilevel Structure of Proteins enables Accurate Prediction of Cold-Start Drug-Target Interactions
Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Beijing Institute of Mathematical Sciences and Applications（北京数学科学与应用研究所）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州先进研究所）；State Key laboratory of Space Network and Communications, Tsinghua University（清华大学空间网络与通信国家重点实验室）
Computation-based Science and Technology Research Center, The Cyprus Institute（塞浦路斯研究所计算科学与技术研究中心）；Faculty of Computing, Engineering and the Built Environment, Birmingham City University（伯明翰城市大学计算、工程与建筑环境学院）
The Influenza virus can be considered as one of the most severe viruses that can infect multiple species with often fatal consequences to the hosts. The Hemagglutinin (HA) gene of the virus can be a target for antiviral drug development realised through accurate identification of its sub-types and possible the targeted hosts. This paper focuses on accurately predicting if an Influenza type A virus can infect specific hosts, and more specifically, Human, Avian and Swine hosts, using only the protein sequence of the HA gene. In more detail, we propose encoding the protein sequences into numerica…
Identifying novel functional protein structures is at the heart of molecular engineering and molecular biology, requiring an often computationally exhaustive search. We introduce the use of a Deep Convolutional Generative Adversarial Network (DCGAN) to classify protein structures based on their functionality by encoding each sample in a grid object structure using three features in each object: the generic atom type, the position atom type, and its occupancy relative to a given atom. We train DCGAN on 3-dimensional (3D) decoy and native protein structures in order to generate and discriminate …
Background: It is necessary and essential to discovery protein function from the novel primary sequences. Wet lab experimental procedures are not only time-consuming, but also costly, so predicting protein structure and function reliably based only on amino acid sequence has significant value. TATA-binding protein (TBP) is a kind of DNA binding protein, which plays a key role in the transcription regulation. Our study proposed an automatic approach for identifying TATA-binding proteins efficiently, accurately, and conveniently. This method would guide for the special protein identification wit…
Centre for Biotechnology and Bioengineering, Universidad de Chile（智利大学生物技术与生物工程中心）；Division of Chemistry and Chemical Engineering, California Institute of Technology（加州理工学院化学与化学工程系）；Max Planck Institute for Dynamics and Self-Organization（马克斯·普朗克动力学与自组织研究所）；Departamento de Ingeniería Química, Biotecnología y Materiales, Facultad de Ciencias, Física y Matemáticas, Universidad de Chile（智利大学科学、物理与数学学院化学工程、生物技术与材料工程系）；Departamento de Ingeniería En Computación, Universidad de Magallanes（麦哲伦大学计算机工程系）；Escuela de Medicina, Universidad de Magallanes（麦哲伦大学医学院）
Predicting the effect of mutations in proteins is one of the most critical challenges in protein engineering; by knowing the effect a substitution of one (or several) residues in the protein's sequence has on its overall properties, could design a variant with a desirable function. New strategies and methodologies to create predictive models are continually being developed. However, those that claim to be general often do not reach adequate performance, and those that aim to a particular task improve their predictive performance at the cost of the method's generality. Moreover, these approache…
University of Missouri（密苏里大学）；Oak Ridge National Laboratory（橡树岭国家实验室）
How and where proteins interface with one another can ultimately impact the proteins' functions along with a range of other biological processes. As such, precise computational methods for protein interface prediction (PIP) come highly sought after as they could yield significant advances in drug discovery and design as well as protein function analysis. However, the traditional benchmark dataset for this task, Docking Benchmark 5 (DB5), contains only a modest 230 complexes for training, validating, and testing different machine learning algorithms. In this work, we expand on a dataset recentl…
Generative modeling for protein engineering is key to solving fundamental problems in synthetic biology, medicine, and material science. We pose protein engineering as an unsupervised sequence generation problem in order to leverage the exponentially growing set of proteins that lack costly, structural annotations. We train a 1.2B-parameter language model, ProGen, on ~280M protein sequences conditioned on taxonomic and keyword tags such as molecular function and cellular component. This provides ProGen with an unprecedented range of evolutionary sequence diversity and allows it to generate wit…
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；SIB Swiss Institute of Bioinformatics（瑞士生物信息学研究所）
Self-supervised neural language models with attention have recently been applied to biological sequence data, advancing structure, function and mutational effect prediction. Some protein language models, including MSA Transformer and AlphaFold's EvoFormer, take multiple sequence alignments (MSAs) of evolutionarily related proteins as inputs. Simple combinations of MSA Transformer's row attentions have led to state-of-the-art unsupervised structural contact prediction. We demonstrate that similarly simple, and universal, combinations of MSA Transformer's column attentions strongly correlate wit…
Predicting protein properties such as solvent accessibility and secondary structure from its primary amino acid sequence is an important task in bioinformatics. Recently, a few deep learning models have surpassed the traditional window based multilayer perceptron. Taking inspiration from the image classification domain we propose a deep convolutional neural network architecture, MUST-CNN, to predict protein properties. This architecture uses a novel multilayer shift-and-stitch (MUST) technique to generate fully dense per-position predictions on protein sequences. Our model is significantly sim…
University of A Coruna（拉科鲁尼亚大学）；University of Bristol（布里斯托尔大学）；Maastricht University（马斯特里赫特大学）
Signaling proteins are an important topic in drug development due to the increased importance of finding fast, accurate and cheap methods to evaluate new molecular targets involved in specific diseases. The complexity of the protein structure hinders the direct association of the signaling activity with the molecular structure. Therefore, the proposed solution involves the use of protein star graphs for the peptide sequence information encoding into specific topological indices calculated with S2SNet tool. The Quantitative Structure-Activity Relationship classification model obtained with Mach…
Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Shanghai Jiao Tong University（上海交通大学）；Peking University（北京大学）；Tsinghua University（清华大学）；HEC Montréal（蒙特利尔高等商学院）
We are now witnessing significant progress of deep learning methods in a variety of tasks (or datasets) of proteins. However, there is a lack of a standard benchmark to evaluate the performance of different methods, which hinders the progress of deep learning in this field. In this paper, we propose such a benchmark called PEER, a comprehensive and multi-task benchmark for Protein sEquence undERstanding. PEER provides a set of diverse protein understanding tasks including protein function prediction, protein localization prediction, protein structure prediction, protein-protein interaction pre…
Computational protein-protein interaction (PPI) prediction techniques can contribute greatly in reducing time, cost and false-positive interactions compared to experimental approaches. Sequence is one of the key and primary information of proteins that plays a crucial role in PPI prediction. Several machine learning approaches have been applied to exploit the characteristics of PPI datasets. However, these datasets greatly influence the performance of predicting models. So, care should be taken on both dataset curation as well as design of predictive models. Here, we have described our submitt…
Tsinghua University（清华大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；Tencent Quantum Lab（腾讯量子实验室）
Protein is linked to almost every life process. Therefore, analyzing the biological structure and property of protein sequences is critical to the exploration of life, as well as disease detection and drug discovery. Traditional protein analysis methods tend to be labor-intensive and time-consuming. The emergence of deep learning models makes modeling data patterns in large quantities of data possible. Interdisciplinary researchers have begun to leverage deep learning methods to model large biological datasets, e.g. using long short-term memory and convolutional neural network for protein sequ…
Composed of amino acid chains that influence how they fold and thus dictating their function and features, proteins are a class of macromolecules that play a central role in major biological processes and are required for the structure, function, and regulation of the body's tissues. Understanding protein functions is vital to the development of therapeutics and precision medicine, and hence the ability to classify proteins and their functions based on measurable features is crucial; indeed, the automatic inference of a protein's properties from its sequence of amino acids, known as its primar…
Proteins are fundamental biological entities mediating key roles in cellular function and disease. This paper introduces a multi-scale graph construction of a protein -- HoloProt -- connecting surface to structure and sequence. The surface captures coarser details of the protein, while sequence as primary component and structure -- comprising secondary and tertiary components -- capture finer details. Our graph encoder then learns a multi-scale representation by allowing each level to integrate the encoding from level(s) below with the graph at that level. We test the learned representation on…
Graph Neural Networks for Protein-Protein Interactions -- A Short Survey
Institute for Infocomm Research (I2R), A*STAR（资讯通信研究院（I2R），新加坡科技研究局）；National University of Singapore (NUS)（新加坡国立大学）；Sun Yat-sen University (SYSU)（中山大学）；Harvard Medical School（哈佛医学院）
Few-shot Protein Fitness Prediction via In-context Learning and Test-time Training
Harvard Medical School（哈佛医学院）；University of Copenhagen（哥本哈根大学）；Novo Nordisk A/S（诺和诺德公司）；Microsoft Research（微软研究院）；Technical University of Denmark（丹麦技术大学）
ResCap-DBP: A Lightweight Residual-Capsule Network for Accurate DNA-Binding Protein Prediction Using Global ProteinBERT Embeddings
Department of Biomedical Engineering, Bangladesh University of Engineering（生物医学工程系，孟加拉国工程大学）；School of Mathematics, Physics and Computing（数学、物理与计算学院）；University of Southern Queensland, Springfield, QLD 4300, Australia（南方昆士兰大学，斯普林菲尔德，QLD 4300，澳大利亚）
本文提出 ResCap-DBP 轻量级残差-胶囊网络，融合全局 ProteinBERT 嵌入直接预测 DNA 结合蛋白，并在多组基准上取得优于现有方法的 AUC 与泛化表现。
Erasmus Medical Center（伊拉斯姆斯医学中心）；IBM Research（IBM研究院）
RNA-binding proteins (RBPs) play crucial roles in many biological processes, e.g. gene regulation. Computational identification of RBP binding sites on RNAs are urgently needed. In particular, RBPs bind to RNAs by recognizing sequence motifs. Thus, fast locating those motifs on RNA sequences is crucial and time-efficient for determining whether the RNAs interact with the RBPs or not. In this study, we present an attention based convolutional neural network, iDeepA, to predict RNA-protein binding sites from raw RNA sequences. We first encode RNA sequences into one-hot encoding. Next, we design …
Protein Thoughts: Interpretable Reasoning with Tree of Thoughts and Embedding-Space Flow Matching for Protein-Protein Interaction Discovery
Department of Statistics and CCAM University of Chicago（统计学系和CCAM大学芝加哥分校）；School of Medicine Stanford University（医学学院斯坦福大学）；Department of Statistics University of Chicago（统计学系芝加哥大学）
本文提出了一种可解释的蛋白质-蛋白质相互作用发现框架，通过显式推理将PPI发现转化为可解释的搜索问题，利用嵌入空间流匹配和树 of 思维搜索方法提升预测精度和可解释性。
University of Sydney（悉尼大学）；University of Technology Sydney（悉尼科技大学）；University of Minnesota（明尼苏达大学）
In this paper, we will show that the recently introduced graphical model: Conditional Random Fields (CRF) provides a template to integrate micro-level information about biological entities into a mathematical model to understand their macro-level behavior. More specifically, we will apply the CRF model to an important classification problem in protein science, namely the secondary structure prediction of proteins based on the observed primary structure. A comparison on benchmark data sets against twenty-eight other methods shows that not only does the CRF model lead to extremely accurate predi…
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）
Protein-ligand complex structures have been utilised to design benchmark machine learning methods that perform important tasks related to drug design such as receptor binding site detection, small molecule docking and binding affinity prediction. However, these methods are usually trained on only ligand bound (or holo) conformations of the protein and therefore are not guaranteed to perform well when the protein structure is in its native unbound conformation (or apo), which is usually the conformation available for a newly identified receptor. A primary reason for this is that the local struc…
Multi-Scale Representation Learning for Protein Fitness Prediction
Mila - Québec AI Institute（米拉 - 魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Harvard Medical School（哈佛医学院）；IBM Research（IBM研究院）；Broad Institute（布罗德研究所）；HEC Montréal（蒙特利尔高等商学院）；CIFAR（加拿大高级研究所）
Motivation: Drug discovery demands rapid quantification of compound-protein interaction (CPI). However, there is a lack of methods that can predict compound-protein affinity from sequences alone with high applicability, accuracy, and interpretability. Results: We present a seamless integration of domain knowledges and learning-based approaches. Under novel representations of structurally-annotated protein sequences, a semi-supervised deep learning model that unifies recurrent and convolutional neural networks has been proposed to exploit both unlabeled and labeled data, for jointly encoding mo…
