DNAGPT:一种用于多种DNA序列分析任务的通用预训练工具
DNAGPT: A Generalized Pre-trained Tool for Versatile DNA Sequence Analysis Tasks
AI总结:
针对预训练语言模型难以适应多种DNA分析任务的问题,提出DNAGPT,利用超2000亿碱基对预训练并增强GPT模型,在多项任务上超越专用模型。
摘要:
预训练大型语言模型在从DNA序列中提取信息方面展现出潜力,但适应各种任务和数据模态仍然是一个挑战。为了解决这一问题,我们提出了DNAGPT,一种在来自所有哺乳动物的超过2000亿碱基对上训练的通用DNA预训练模型。通过用二分类任务(DNA序列顺序)、数值回归任务(鸟嘌呤-胞嘧啶含量预测)和全面的标记语言增强经典GPT模型,DNAGPT可以处理多种DNA分析任务,同时处理序列和数值数据。我们对基因组信号和区域识别、mRNA丰度回归以及人工基因组生成任务的评估表明,与为特定下游任务设计的现有模型相比,DNAGPT表现出优越的性能,这得益于使用新设计的模型结构进行的预训练。
英文摘要:
Pre-trained large language models demonstrate potential in extracting information from DNA sequences, yet adapting to a variety of tasks and data modalities remains a challenge. To address this, we propose DNAGPT, a generalized DNA pre-training model trained on over 200 billion base pairs from all mammals. By enhancing the classic GPT model with a binary classification task (DNA sequence order), a numerical regression task (guanine-cytosine content prediction), and a comprehensive token language, DNAGPT can handle versatile DNA analysis tasks while processing both sequence and numerical data. Our evaluation of genomic signal and region recognition, mRNA abundance regression, and artificial genomes generation tasks demonstrates DNAGPT's superior performance compared to existing models designed for specific downstream tasks, benefiting from pre-training using the newly designed model structure.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。