arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.12459cs.LGcs.CE
收藏

HELM:面向mRNA语言建模的分层编码

HELM: Hierarchical Encoding for mRNA Language Modeling

University of Central Florida(中佛罗里达大学);Johnson & Johnson Innovative Medicine(强生创新医药)

Mehdi Yazdani-Jahromi, Mangal Prakash, Tommaso Mansi, Artem Moskalev, Rui Liao

AI总结:

本文提出HELM预训练策略,通过将mRNA密码子分层结构纳入语言模型训练并基于同义性调节损失函数,在多个下游性质预测和抗体区域注释任务上平均提升约8%,同时改善了mRNA序列生成质量。

摘要:

信使RNA(mRNA)在蛋白质合成中起着至关重要的作用,其密码子结构直接影响生物学特性。虽然语言模型(LMs)在分析生物序列方面已展现出潜力,但现有方法未能考虑mRNA密码子结构的分层性质。我们提出了面向mRNA语言建模的分层编码(HELM),这是一种新颖的预训练策略,将密码子级别的分层结构纳入语言模型训练。HELM根据密码子同义性调节损失函数,使模型的学习过程与mRNA序列的生物学现实保持一致。我们在多样的mRNA数据集和任务上评估HELM,结果表明,在七个不同的下游性质预测任务和一个抗体区域注释任务上,HELM平均比标准语言模型预训练以及现有基础模型基线高出约8%。此外,HELM增强了语言模型的生成能力,与非分层基线相比,能够产生更多样的mRNA序列,并且更好地与底层真实数据分布对齐。

英文摘要:

Messenger RNA (mRNA) plays a crucial role in protein synthesis, with its codon structure directly impacting biological properties. While Language Models (LMs) have shown promise in analyzing biological sequences, existing approaches fail to account for the hierarchical nature of mRNA's codon structure. We introduce Hierarchical Encoding for mRNA Language Modeling (HELM), a novel pre-training strategy that incorporates codon-level hierarchical structure into language model training. HELM modulates the loss function based on codon synonymity, aligning the model's learning process with the biological reality of mRNA sequences. We evaluate HELM on diverse mRNA datasets and tasks, demonstrating that HELM outperforms standard language model pre-training as well as existing foundation model baselines on seven diverse downstream property prediction tasks and an antibody region annotation tasks on average by around 8%. Additionally, HELM enhances the generative capabilities of language model, producing diverse mRNA sequences that better align with the underlying true data distribution compared to non-hierarchical baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑