PoET：将蛋白质家族作为序列的序列的生成模型
PoET: A generative model of protein families as sequences-of-sequences
AI总结：
提出蛋白质进化Transformer（PoET），一种将蛋白质家族作为序列的序列进行建模的自回归生成模型，通过独特的Transformer层实现跨家族迁移学习与长上下文外推，在变体功能预测和新蛋白质生成上优于现有模型。
摘要：
生成式蛋白质语言模型是设计具有所需功能的新蛋白质的自然方法。然而，当前的模型要么难以引导其生成特定感兴趣的蛋白质家族的蛋白质，要么必须在来自特定感兴趣家族的大型多序列比对（MSA）上进行训练，这使其无法受益于跨家族的迁移学习。为了解决这个问题，我们提出了蛋白质进化Transformer（PoET），这是一种整个蛋白质家族的自回归生成模型，它学习将相关蛋白质集合作为序列的序列生成，跨越数千万个天然蛋白质序列簇。PoET可用作检索增强语言模型，以任何感兴趣的蛋白质家族为条件生成和评分任意修改，并且可以从短上下文长度进行外推，即使对于小家族也能很好地泛化。这是通过一个独特的Transformer层实现的；我们在序列内按顺序对标记进行建模，同时在序列之间以顺序不变的方式进行关注，这允许PoET扩展到超出训练期间使用的上下文长度。在深度突变扫描数据集上的大量实验中，我们表明，在所有MSA深度的蛋白质的变体功能预测方面，PoET优于现有的蛋白质语言模型和进化序列模型。我们还展示了PoET可控地生成新蛋白质序列的能力。
英文摘要：
Generative protein language models are a natural way to design new proteins with desired functions. However, current models are either difficult to direct to produce a protein from a specific family of interest, or must be trained on a large multiple sequence alignment (MSA) from the specific family of interest, making them unable to benefit from transfer learning across families. To address this, we propose $\textbf{P}$r$\textbf{o}$tein $\textbf{E}$volutionary $\textbf{T}$ransformer (PoET), an autoregressive generative model of whole protein families that learns to generate sets of related proteins as sequences-of-sequences across tens of millions of natural protein sequence clusters. PoET can be used as a retrieval-augmented language model to generate and score arbitrary modifications conditioned on any protein family of interest, and can extrapolate from short context lengths to generalize well even for small families. This is enabled by a unique Transformer layer; we model tokens sequentially within sequences while attending between sequences order invariantly, allowing PoET to scale to context lengths beyond those used during training. In extensive experiments on deep mutational scanning datasets, we show that PoET outperforms existing protein language models and evolutionary sequence models for variant function prediction across proteins of all MSA depths. We also demonstrate PoET's ability to controllably generate new protein sequences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
