arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.08334cs.LGcs.AI
收藏

化学大语言模型能否从消息传递中获益

Could Chemical LLMs benefit from Message Passing

Department of Computer Science(计算机科学系);ETH Zurich(苏黎世联邦理工学院)

Jiaqing Xie, Ziheng Chi

AI总结:

本研究针对分子结构与文本表示间双向交互研究不足的问题,提出对比学习和融合两种信息整合策略,探究消息传递神经网络与化学大语言模型的结合效果,发现其在小分子图任务上优于基线,大规模图上无提升。

摘要:

预训练语言模型(LM)在处理分子文本方面展现出显著能力,与此同时,消息传递神经网络(MPNNs)在分子科学领域表现出鲁棒性与通用性。尽管已有这些进展,我们发现针对分子结构与其对应文本表示之间双向交互的研究仍较为有限。因此,本文提出两种策略以评估信息整合能否提升性能:对比学习,即利用MPNN监督LM的训练;以及融合策略,即利用来自两种模型的信息。我们的实证分析表明,当应用于较小分子图时,这些整合方法相比基线模型表现出更优性能,而在大规模图上,这些整合方法并未带来性能提升。

英文摘要:

Pretrained language models (LMs) showcase significant capabilities in processing molecular text, while concurrently, message passing neural networks (MPNNs) demonstrate resilience and versatility in the domain of molecular science. Despite these advancements, we find there are limited studies investigating the bidirectional interactions between molecular structures and their corresponding textual representations. Therefore, in this paper, we propose two strategies to evaluate whether an information integration can enhance the performance: contrast learning, which involves utilizing an MPNN to supervise the training of the LM, and fusion, which exploits information from both models. Our empirical analysis reveals that the integration approaches exhibit superior performance compared to baselines when applied to smaller molecular graphs, while these integration approaches do not yield performance enhancements on large scale graphs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑