arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.24022q-bio.QMcs.AIcs.CLcs.LG
收藏

SFM-Protein:面向高级蛋白质序列表征的整合式共进化预训练

SFM-Protein: Integrative Co-evolutionary Pre-training for Advanced Protein Sequence Representation

Microsoft Research(微软研究院);Zhejiang University(浙江大学);Huazhong University of Science and Technology(华中科技大学);Tsinghua University(清华大学)

Liang He, Peiran Jin, Yaosen Min, Shufang Xie, Lijun Wu, Tao Qin, Xiaozhuan Liang, Kaiyuan Gao, Yuliang Jiang, Tie-Yan Liu

AI总结:

针对传统蛋白质基础模型难捕捉共进化信息的问题,本研究提出SFM-Protein预训练策略,强化氨基酸残基相互作用以提取长短程共进化特征,在多下游任务中优于同等规模ESM等基线,推动序列蛋白质建模发展。

摘要:

蛋白质是生物系统的核心组成,其功能发挥与三维结构存在复杂关联。解析蛋白质结构与氨基酸序列的对应关系,始终是蛋白质建模领域的核心难题。传统蛋白质基础模型依托海量无标注数据集预训练取得效果,但往往难以捕捉关键的共进化信息,而这正是进化类方法的优势所在。本研究提出一种面向蛋白质基础模型的新型预训练策略,重点强化氨基酸残基间的相互作用,以提升从序列数据中提取短程及长程共进化特征的能力。该模型在大规模蛋白质序列数据集上训练后展现出更强的泛化能力,在多种下游任务中表现优于同等规模的现有基线模型(包括ESM模型)。实验结果证实该模型可有效整合共进化信息,为基于序列的蛋白质建模研究迈出重要一步。

英文摘要:

Proteins, essential to biological systems, perform functions intricately linked to their three-dimensional structures. Understanding the relationship between protein structures and their amino acid sequences remains a core challenge in protein modeling. While traditional protein foundation models benefit from pre-training on vast unlabeled datasets, they often struggle to capture critical co-evolutionary information, which evolutionary-based methods excel at. In this study, we introduce a novel pre-training strategy for protein foundation models that emphasizes the interactions among amino acid residues to enhance the extraction of both short-range and long-range co-evolutionary features from sequence data. Trained on a large-scale protein sequence dataset, our model demonstrates superior generalization ability, outperforming established baselines of similar size, including the ESM model, across diverse downstream tasks. Experimental results confirm the model's effectiveness in integrating co-evolutionary information, marking a significant step forward in protein sequence-based modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑