arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.09043cs.AI
收藏

视觉语言模型并非你所需要的一切:分子语言模型的增强策略

Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models

KAIST(韩国科学技术院);UIUC(伊利诺伊大学厄巴纳-香槟分校)

Namkyeong Lee, Siddhartha Laghuvarapu, Chanyoung Park, Jimeng Sun

AI总结:

针对分子语言模型数据稀缺与专业知识缺失的挑战,提出AMOLE,通过结构相似性保持损失和专业知识重构损失增强分子-文本对并转移知识,在多项下游任务中表现优越,助力药物发现。

摘要:

近期,研究人员对通过分子语言模型理解分子及其文本描述产生了日益浓厚的兴趣。然而,尽管已有一些早期充满希望的发展,MoLM的进步仍显著落后于视觉语言模型。这是因为在MoLM领域,除了VLM之外还存在独特的挑战,原因在于:1)分子-文本配对数据数量有限;2)专家专注于特定领域而导致了专业知识缺失。为此,我们提出了AMOLE,它1)通过结构相似性保持损失来增强分子-文本对,2)在分子之间转移专业知识。具体而言,AMOLE通过一种新颖的结构相似性保持损失,在结构相似的分子间共享描述,从而丰富了分子-文本对。此外,我们提出了一种专业知识重构损失,将知识从具备大量专业知识的分子转移到专业知识较少的分子上。在多个下游任务上的大量实验证明了AMOLE在理解分子及其描述方面的优越性,突显了其在现实世界药物发现中的应用潜力。AMOLE的源代码可在 https://github.com/Namkyeong/AMOLE 获取。

英文摘要:

Recently, there has been a growing interest among researchers in understanding molecules and their textual descriptions through molecule language models (MoLM). However, despite some early promising developments, the advancement of MoLM still trails significantly behind that of vision language models (VLM). This is because unique challenges exist apart from VLM in the field of MoLM due to 1) a limited amount of molecule-text paired data and 2) missing expertise that occurred due to the specialized areas of focus among the experts. To this end, we propose AMOLE, which 1) augments molecule-text pairs with structural similarity preserving loss, and 2) transfers the expertise between the molecules. Specifically, AMOLE enriches molecule-text pairs by sharing descriptions among structurally similar molecules with a novel structural similarity preserving loss. Moreover, we propose an expertise reconstruction loss to transfer knowledge from molecules that have extensive expertise to those with less expertise. Extensive experiments on various downstream tasks demonstrate the superiority of AMOLE in comprehending molecules and their descriptions, highlighting its potential for application in real-world drug discovery. The source code for AMOLE is available at https://github.com/Namkyeong/AMOLE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑