ProtST：蛋白质序列与生物医学文本的多模态学习
ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts
AI总结：
本文提出ProtST框架，通过构建ProtDescribe数据集并设计三类预训练任务，将生物医学文本与蛋白质序列进行多模态对齐，从而增强蛋白质语言模型的功能表示能力，并在监督学习、零样本分类和功能检索任务上验证其优越性。
摘要：
当前的蛋白质语言模型（PLMs）主要基于序列学习蛋白质表示，从而能够很好地捕捉协同进化信息，但无法显式获取蛋白质功能，而蛋白质功能正是蛋白质表示学习的最终目标。幸运的是，对于许多蛋白质而言，其文本属性描述是可获得的，其中也描述了它们的各种功能。受这一事实启发，我们首先构建了ProtDescribe数据集，以蛋白质功能及其他重要属性的文本描述来增强蛋白质序列。基于该数据集，我们提出了ProtST框架，通过生物医学文本来增强蛋白质序列的预训练与理解。在预训练过程中，我们设计了三类任务，即单模态掩码预测、多模态表示对齐和多模态掩码预测，以不同粒度的蛋白质属性信息增强PLM，同时保持PLM原有的表示能力。在下游任务上，ProtST同时支持监督学习和零样本预测。我们在多种表示学习基准上验证了由ProtST增强的PLM相较于以往PLM的优越性。在零样本设置下，我们展示了ProtST在零样本蛋白质分类上的有效性，并且ProtST还能够在没有任何功能注释的情况下从大规模数据库中实现功能性蛋白质检索。
英文摘要：
Current protein language models (PLMs) learn protein representations mainly based on their sequences, thereby well capturing co-evolutionary information, but they are unable to explicitly acquire protein functions, which is the end goal of protein representation learning. Fortunately, for many proteins, their textual property descriptions are available, where their various functions are also described. Motivated by this fact, we first build the ProtDescribe dataset to augment protein sequences with text descriptions of their functions and other important properties. Based on this dataset, we propose the ProtST framework to enhance Protein Sequence pre-training and understanding by biomedical Texts. During pre-training, we design three types of tasks, i.e., unimodal mask prediction, multimodal representation alignment and multimodal mask prediction, to enhance a PLM with protein property information with different granularities and, at the same time, preserve the PLM's original representation power. On downstream tasks, ProtST enables both supervised learning and zero-shot prediction. We verify the superiority of ProtST-induced PLMs over previous ones on diverse representation learning benchmarks. Under the zero-shot setting, we show the effectiveness of ProtST on zero-shot protein classification, and ProtST also enables functional protein retrieval from a large-scale database without any function annotation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
