arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.06150cs.CLphysics.chem-phq-bio.BM
收藏

LipidBERT:一种在METiS从头脂质库上预训练的脂质语言模型

LipidBERT: A Lipid Language Model Pre-trained on METiS de novo Lipid Library

METiS Pharmaceuticals(METiS制药公司)

Tianhao Yu, Cai Yao, Zhuorui Sun, Feng Shi, Lin Zhang, Kangjie Lyu, Xuan Bai, Andong Liu, Xicheng Zhang, Jiali Zou, Wenshou Wang, Chris Lai, Kai Wang

AI总结:

本文提出LipidBERT,利用METiS内部1000万虚拟脂质库进行BERT式预训练并结合LNP湿实验数据微调,实现最先进的LNP性质预测,展示了干湿实验整合在脂质筛选中的价值。

摘要:

在本研究中,我们通过METiS内部的从头脂质生成算法和脂质虚拟筛选技术,生成并维护了一个包含1000万个虚拟脂质的数据库。这些虚拟脂质作为预训练、脂质表征学习和下游任务知识迁移的语料库,最终实现了最先进的LNP性质预测性能。我们提出了LipidBERT,这是一种类似BERT的模型,使用掩码语言模型(MLM)和多种辅助任务进行预训练。此外,我们比较了LipidBERT和PhatGPT(我们类似GPT的脂质生成模型)生成的嵌入在下游任务上的性能。所提出的双语LipidBERT模型以两种语言运行:可电离脂质预训练语言,使用内部干实验脂质结构;以及LNP微调语言,利用内部LNP湿实验数据。这种双重能力使LipidBERT成为未来筛选任务的关键AI过滤器,包括新版本的METiS从头脂质库,更重要的是,用于器官靶向LNP的体内测试候选物。据我们所知,这是首次成功证明预训练语言模型在虚拟脂质上的能力及其在使用湿实验数据的下游任务中的有效性。这项工作展示了METiS内部从头脂质库的巧妙利用以及干湿实验整合的力量。

英文摘要:

In this study, we generate and maintain a database of 10 million virtual lipids through METiS's in-house de novo lipid generation algorithms and lipid virtual screening techniques. These virtual lipids serve as a corpus for pre-training, lipid representation learning, and downstream task knowledge transfer, culminating in state-of-the-art LNP property prediction performance. We propose LipidBERT, a BERT-like model pre-trained with the Masked Language Model (MLM) and various secondary tasks. Additionally, we compare the performance of embeddings generated by LipidBERT and PhatGPT, our GPT-like lipid generation model, on downstream tasks. The proposed bilingual LipidBERT model operates in two languages: the language of ionizable lipid pre-training, using in-house dry-lab lipid structures, and the language of LNP fine-tuning, utilizing in-house LNP wet-lab data. This dual capability positions LipidBERT as a key AI-based filter for future screening tasks, including new versions of METiS de novo lipid libraries and, more importantly, candidates for in vivo testing for orgran-targeting LNPs. To the best of our knowledge, this is the first successful demonstration of the capability of a pre-trained language model on virtual lipids and its effectiveness in downstream tasks using web-lab data. This work showcases the clever utilization of METiS's in-house de novo lipid library as well as the power of dry-wet lab integration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑