arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.07747cs.LGq-bio.QM
收藏

面向长蛋白质序列分析的ESM2架构扩展:长序列与量化方法

Scaling Up ESM2 Architectures for Long Protein Sequences Analysis: Long and Quantized Approaches

Institute of Computing, University of Campinas(坎皮纳斯大学计算研究所)

Gabriel Bianchin de Oliveira, Helio Pedrini, Zanoni Dias

AI总结:

针对ESM2蛋白质序列模型仅支持1022个氨基酸输入的局限,本文提出其长序列与量化版本,将输入长度上限提升至2048个氨基酸,为长蛋白质序列分析提供更优的基础架构。

摘要:

基于Transformer架构的各类方法已在自然语言处理(NLP)领域取得了当前最优结果。受此成功启发,研究者已针对生物学等其他类型数据(尤其是蛋白质序列)提出了众多架构,其中值得关注的是在数十亿条蛋白质上预训练的ESM2架构,它是该领域多种当前最优方法的基础。但ESM2架构存在输入尺寸限制,仅支持最多1022个氨基酸,处理超出该长度的序列时必须使用预处理技术。本文提出了ESM2架构的长序列版本与量化版本,将输入尺寸限制提升一倍至2048个氨基酸。

英文摘要:

Various approaches utilizing Transformer architectures have achieved state-of-the-art results in Natural Language Processing (NLP). Based on this success, numerous architectures have been proposed for other types of data, such as in biology, particularly for protein sequences. Notably among these are the ESM2 architectures, pre-trained on billions of proteins, which form the basis of various state-of-the-art approaches in the field. However, the ESM2 architectures have a limitation regarding input size, restricting it to 1,022 amino acids, which necessitates the use of preprocessing techniques to handle sequences longer than this limit. In this paper, we present the long and quantized versions of the ESM2 architectures, doubling the input size limit to 2,048 amino acids.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑