重寻中心法则:利用大型语言模型中的多语言迁移
Find Central Dogma Again: Leveraging Multilingual Transfer in Large Language Models
AI总结:
本文训练并微调多语言GPT-2,将DNA-蛋白质匹配建模为二分类任务,证明自然语言迁移可使其在无先验知识时零样本重寻中心法则。
摘要:
近年来,大型语言模型(LLM)已在各类生物序列分析任务中取得最先进结果,例如序列分类、结构预测和功能预测。与人工智能在其他科学领域中的进展类似,对生物LLM更深入的研究已开始关注使用这些模型重新发现重要的既有生物学规律,或揭示生物序列中全新的模式。本研究利用类似GPT的LLM,借助语言迁移能力重新发现中心法则的遗传密码规则。在实验设计中,我们将中心法则转化为一个DNA序列与蛋白质序列对齐的二分类问题,其中正例是相互匹配的DNA和蛋白质序列,负例是不匹配的序列对。我们首先使用一个包含蛋白质序列、DNA序列以及英语和中文等语言序列的数据集,从头训练了一个GPT-2模型。随后,我们使用来自PAWS-X的自然语言句子相似度判断数据集对该模型进行微调。在用于DNA和蛋白质序列对齐判断的数据集上测试时,经过微调的模型取得了81%的分类准确率。本研究还分析了促成这种零样本能力的因素,包括模型训练稳定性和训练数据类型。本研究表明,LLM能够通过自然语言能力的迁移,并且仅依靠对序列本身的分析,在不具备中心法则先验知识的情况下重新发现中心法则。本研究在自然语言和遗传语言之间架起桥梁,为人工智能驱动的生物学研究打开了一扇新大门。
英文摘要:
In recent years, large language models (LLMs) have achieved state-of-the-art results in various biological sequence analysis tasks, such as sequence classification, structure prediction, and function prediction. Similar to advancements in AI for other scientific fields, deeper research into biological LLMs has begun to focus on using these models to rediscover important existing biological laws or uncover entirely new patterns in biological sequences. This study leverages GPT-like LLMs to utilize language transfer capabilities to rediscover the genetic code rules of the central dogma. In our experimental design, we transformed the central dogma into a binary classification problem of aligning DNA sequences with protein sequences, where positive examples are matching DNA and protein sequences, and negative examples are non-matching pairs. We first trained a GPT-2 model from scratch using a dataset comprising protein sequences, DNA sequences, and sequences from languages such as English and Chinese. Subsequently, we fine-tuned the model using the natural language sentences similarity judgment dataset from PAWS-X. When tested on a dataset for DNA and protein sequence alignment judgment, the fine-tuned model achieved a classification accuracy of 81%. The study also analyzed factors contributing to this zero-shot capability, including model training stability and types of training data. This research demonstrates that LLMs can, through the transfer of natural language capabilities and solely relying on the analysis of sequences themselves, rediscover the central dogma without prior knowledge of it. This study bridges natural language and genetic language, opening a new door for AI-driven biological research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。