ProtLLM：一种以蛋白质为词预训练的交叉蛋白质-语言大语言模型
ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training
AI总结：
ProtLLM提出动态蛋白质挂载机制和以蛋白质为词的语言建模，利用大规模交错蛋白质-文本数据集InterPT预训练，在蛋白质中心任务上超越专用基线，并具备蛋白质-语言任务的零样本与上下文学习能力。
摘要：
我们提出了ProtLLM，一种用于蛋白质中心任务和蛋白质-语言任务的通用跨模态大语言模型（LLM）。ProtLLM具有独特的动态蛋白质挂载机制，使其能够处理自然语言文本与任意数量蛋白质交织的复杂输入。此外，我们提出了以蛋白质为词的语言建模方法来训练ProtLLM。通过开发专门的蛋白质词汇表，我们使模型不仅能够预测自然语言，还能从大量候选蛋白质中预测蛋白质。另外，我们构建了一个大规模的交错蛋白质-文本数据集，命名为InterPT，用于预训练。该数据集全面涵盖了（1）结构化数据源，如蛋白质注释，和（2）非结构化数据源，如生物学研究论文，从而赋予ProtLLM理解蛋白质的关键知识。我们在经典的监督蛋白质中心任务上评估了ProtLLM，并探索了其新颖的蛋白质-语言应用。实验结果表明，ProtLLM不仅在蛋白质中心任务上取得了优于蛋白质专门基线的性能，而且在蛋白质-语言任务上还展现出了零样本和上下文学习能力。
英文摘要：
We propose ProtLLM, a versatile cross-modal large language model (LLM) for both protein-centric and protein-language tasks. ProtLLM features a unique dynamic protein mounting mechanism, enabling it to handle complex inputs where the natural language text is interspersed with an arbitrary number of proteins. Besides, we propose the protein-as-word language modeling approach to train ProtLLM. By developing a specialized protein vocabulary, we equip the model with the capability to predict not just natural language but also proteins from a vast pool of candidates. Additionally, we construct a large-scale interleaved protein-text dataset, named InterPT, for pre-training. This dataset comprehensively encompasses both (1) structured data sources like protein annotations and (2) unstructured data sources like biological research papers, thereby endowing ProtLLM with crucial knowledge for understanding proteins. We evaluate ProtLLM on classic supervised protein-centric tasks and explore its novel protein-language applications. Experimental results demonstrate that ProtLLM not only achieves superior performance against protein-specialized baselines on protein-centric tasks but also induces zero-shot and in-context learning capabilities on protein-language tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
