蛋白质语言模型的自我蒸馏微调提升了蛋白质设计的通用性
Self Distillation Fine-Tuning of Protein Language Models Improves Versatility in Protein Design
AI总结：
通过自我蒸馏微调提升蛋白质语言模型的通用性，生成更稳定和功能性的蛋白质序列。
摘要：
监督微调（SFT）是将大语言模型适应到专业领域的一种标准方法，但其在蛋白质序列建模和蛋白质语言模型（PLMs）中的应用仍较为随意。这在一定程度上是因为获得高质量注释数据比获得自然语言数据更困难。我们提出了一种简单且通用的方法，用于快速微调PLMs，旨在提高生成蛋白质序列的准确性、可靠性和新颖性。与现有方法需要昂贵的预编译实验数据集不同，我们的方法利用PLM本身，整合一个轻量级的整理流程和领域特定的过滤器来构建高质量的训练数据。这些过滤器可以独立地优化PLM的输出并识别候选物进行体外评估；当与SFT结合时，它们使PLMs能够生成更稳定和功能性的酶，同时扩展对蛋白质序列空间的探索，超越自然变异。尽管我们的方法对所选蛋白质语言模型（PLM）和蛋白质系统的选择都是无关的，但我们通过基因组规模的PLM（GenSLM）应用于色氨酸合成酶家族展示了其有效性。监督微调的模型生成的序列不仅更加新颖，而且在目标设计约束和新兴蛋白质属性测量方面也表现出改进的特性。
英文摘要：
Supervised fine-tuning (SFT) is a standard approach for adapting large language models to specialized domains, yet its application to protein sequence modeling and protein language models (PLMs) remains ad hoc. This is in part because high-quality annotated data are far more difficult to obtain for proteins than for natural language. We present a simple and general recipe for fast SFT of PLMs, designed to improve the fidelity, reliability, and novelty of generated protein sequences. Unlike existing approaches that require costly precompiled experimental datasets for SFT, our method leverages the PLM itself, integrating a lightweight curation pipeline with domain-specific filters to construct high-quality training data. These filters can independently refine a PLM's output and identify candidates for in vitro evaluation; when combined with SFT, they enable PLMs to generate more stable and functional enzymes, while expanding exploration into protein sequence space beyond natural variants. Although our approach is agnostic to both the choice of protein language model (PLM) and the protein system, we demonstrate its effectiveness with a genome-scale PLM (GenSLM) applied to the tryptophan synthase enzyme family. The supervised fine-tuned model generates sequences that are not only more novel but also display improved characteristics across both targeted design constraints and emergent protein property measures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
