PhyloTransformer:一种基于多头自注意机制的判别式突变预测模型
PhyloTransformer: A Discriminative Model for Mutation Prediction Based on a Multi-head Self-attention Mechanism
摘要:
严重急性呼吸综合征冠状病毒2(SARS-CoV-2)已导致持续的大流行,截至2021年10月19日,感染了2.19亿人,死亡率为3.6%。自然选择可以产生具有改进的适应性优势的有利突变;然而,已鉴定的冠状病毒可能是冰山一角,随着时间的推移,可能会出现更致命的变异。迫切需要了解新出现的VOC的模式并预测可能导致功能获得或免疫逃逸的突变。在这里,我们开发了PhyloTransformer,这是一种基于Transformer的判别模型,它采用多头自我注意机制来模拟可能导致病毒繁殖优势的基因突变。为了识别每个输入序列的元素之间的复杂依赖关系,PhyloTransformer利用了先进的建模技术,包括Performer的一种新的通过正正交随机特征的快速注意方法(FAVOR+),以及来自Transformers的双向编码器表示(BERT)的掩蔽语言模型(MLM)。PhyloTransformer使用从全球流感数据共享倡议(GISAID)数据库检索的1,765,297个基因序列进行训练。首先,我们使用广泛的基线模型比较了新突变和新组合的预测准确性;我们发现PhyloTransformer优于具有统计学显著性的每种基线方法。其次,我们检查了受体结合基序(RBM)的每个核苷酸突变的预测,我们发现我们的预测是精确和准确的。第三,我们预测了N-糖基化位点的修饰,以鉴定与糖基化改变相关的突变,这些突变可能在病毒进化过程中受到青睐。我们预计PhyloTransformer可以指导主动疫苗设计,以有效靶向未来的SARS-CoV-2变体。
英文摘要:
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused an ongoing pandemic infecting 219 million people as of 10/19/21, with a 3.6% mortality rate. Natural selection can generate favorable mutations with improved fitness advantages; however, the identified coronaviruses may be the tip of the iceberg, and potentially more fatal variants of concern (VOCs) may emerge over time. Understanding the patterns of emerging VOCs and forecasting mutations that may lead to gain of function or immune escape is urgently required. Here we developed PhyloTransformer, a Transformer-based discriminative model that engages a multi-head self-attention mechanism to model genetic mutations that may lead to viral reproductive advantage. In order to identify complex dependencies between the elements of each input sequence, PhyloTransformer utilizes advanced modeling techniques, including a novel Fast Attention Via positive Orthogonal Random features approach (FAVOR+) from Performer, and the Masked Language Model (MLM) from Bidirectional Encoder Representations from Transformers (BERT). PhyloTransformer was trained with 1,765,297 genetic sequences retrieved from the Global Initiative for Sharing All Influenza Data (GISAID) database. Firstly, we compared the prediction accuracy of novel mutations and novel combinations using extensive baseline models; we found that PhyloTransformer outperformed every baseline method with statistical significance. Secondly, we examined predictions of mutations in each nucleotide of the receptor binding motif (RBM), and we found our predictions were precise and accurate. Thirdly, we predicted modifications of N-glycosylation sites to identify mutations associated with altered glycosylation that may be favored during viral evolution. We anticipate that PhyloTransformer may guide proactive vaccine design for effective targeting of future SARS-CoV-2 variants.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。