arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.07930q-bio.BMcs.LG
收藏

Token-Mol 1.0: 基于大语言模型的Token化药物设计

Token-Mol 1.0: Tokenized drug design with large language model

Zhejiang University(浙江大学);Huawei Technologies Co., Ltd.(华为技术有限公司);University of Washington(华盛顿大学);Tongji University(同济大学)
Jike Wang, Rui Qin, Mingyang Wang, Meijing Fang, Yangyang Zhang, Yuchen Zhu, Qun Su, Qiaolin Gou, Chao Shen, Odin Zhang, Zhenxing Wu, Dejun Jiang, Xujun Zhang, … 展开作者

Jike Wang, Rui Qin, Mingyang Wang, Meijing Fang, Yangyang Zhang, Yuchen Zhu, Qun Su, Qiaolin Gou, Chao Shen, Odin Zhang, Zhenxing Wu, Dejun Jiang, Xujun Zhang, Huifeng Zhao, Xiaozhe Wan, Zhourui Wu, Liwei Liu, Yu Kang, Chang-Yu Hsieh, Tingjun Hou

AI总结:

针对现有大语言模型药物设计缺乏3D结构理解能力的问题,提出纯token化3D药物设计模型Token-Mol,结合GCE损失、微调和强化学习,在多下游任务上性能优异,为通用AI药物设计奠定基础。

摘要:

近年来,利用基于序列的大语言模型(LLM)开展药物设计的研究受到了广泛关注。然而,当前大多数LLM在药物发现中的应用都缺乏理解三维(3D)结构的能力,从而限制了其在明确涉及分子构象的任务中的有效性。本研究提出了Token-Mol,一种纯token化的3D药物设计模型。该模型将所有分子信息(包括2D和3D结构以及分子性质数据)编码为token,将药物发现中的分类和回归任务转化为概率预测问题,从而实现统一范式下的学习。Token-Mol基于transformer解码器架构构建,并采用随机因果掩码技术进行训练。此外,我们提出了高斯交叉熵(GCE)损失函数,以克服回归任务中的挑战,显著提升了LLM学习连续数值的能力。通过微调与强化学习(RL)相结合的方式,Token-Mol在包括基于口袋的分子生成、构象生成和分子性质预测在内的多种下游任务上,取得了与现有任务特定方法相当甚至更优的性能。与现有的分子预训练模型相比,Token-Mol在处理药物设计所需的更广泛下游任务方面表现出更优异的能力。值得注意的是,与同类纯token化方法相比,我们的方法将回归任务的准确率提升了约30%。Token-Mol突破了纯token化模型的精度限制,具备与ChatGPT等通用模型无缝集成的潜力,为开发通用人工智能药物设计模型铺平了道路,助力专家快速、高质量地开展药物设计。

英文摘要:

Significant interests have recently risen in leveraging sequence-based large language models (LLMs) for drug design. However, most current applications of LLMs in drug discovery lack the ability to comprehend three-dimensional (3D) structures, thereby limiting their effectiveness in tasks that explicitly involve molecular conformations. In this study, we introduced Token-Mol, a token-only 3D drug design model. This model encodes all molecular information, including 2D and 3D structures, as well as molecular property data, into tokens, which transforms classification and regression tasks in drug discovery into probabilistic prediction problems, thereby enabling learning through a unified paradigm. Token-Mol is built on the transformer decoder architecture and trained using random causal masking techniques. Additionally, we proposed the Gaussian cross-entropy (GCE) loss function to overcome the challenges in regression tasks, significantly enhancing the capacity of LLMs to learn continuous numerical values. Through a combination of fine-tuning and reinforcement learning (RL), Token-Mol achieves performance comparable to or surpassing existing task-specific methods across various downstream tasks, including pocket-based molecular generation, conformation generation, and molecular property prediction. Compared to existing molecular pre-trained models, Token-Mol exhibits superior proficiency in handling a wider range of downstream tasks essential for drug design. Notably, our approach improves regression task accuracy by approximately 30% compared to similar token-only methods. Token-Mol overcomes the precision limitations of token-only models and has the potential to integrate seamlessly with general models such as ChatGPT, paving the way for the development of a universal artificial intelligence drug design model that facilitates rapid and high-quality drug design by experts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑