arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.06010cs.LGq-bio.BM
收藏

GNN预训练有助于分子表征吗?

Does GNN Pretraining Help Molecular Representation?

Google Cloud AI Research(谷歌云人工智能研究院);Google Research(谷歌研究院)

Ruoxi Sun, Hanjun Dai, Adams Wei Yu

摘要:

使用图神经网络(GNN)提取分子的信息表示在AI驱动的药物发现中至关重要。最近,图研究社区一直在试图复制自然语言处理中自我监督预训练的成功,并取得了一些成功。然而,我们发现在许多情况下,对小分子数据进行自我监督预训练所带来的好处可以忽略不计。我们对GNN预训练的关键组成部分进行了彻底的消融研究,包括预训练目标,数据分割方法,输入特征,预训练数据集规模和GNN架构,以了解它们如何影响下游任务的准确性。我们的第一个重要发现是,在许多情况下,自监督图预训练并不总是比非预训练方法具有统计上的显著优势。其次,虽然通过额外的监督预训练可以观察到明显的改善,但随着更丰富的特征或更平衡的数据分割,这种改善可能会减少。第三,超参数对下游任务准确率的影响可能大于预训练任务的选择,特别是当下游任务规模较小时。最后,我们提供了一些小分子预训练方法的复杂性可能不足的地方,然后是不同预训练数据集上的经验证据。

英文摘要:

Extracting informative representations of molecules using Graph neural networks (GNNs) is crucial in AI-driven drug discovery. Recently, the graph research community has been trying to replicate the success of self-supervised pretraining in natural language processing, with several successes claimed. However, we find the benefit brought by self-supervised pretraining on small molecular data can be negligible in many cases. We conduct thorough ablation studies on the key components of GNN pretraining, including pretraining objectives, data splitting methods, input features, pretraining dataset scales, and GNN architectures, to see how they affect the accuracy of the downstream tasks. Our first important finding is, self-supervised graph pretraining do not always have statistically significant advantages over non-pretraining methods in many settings. Secondly, although noticeable improvement can be observed with additional supervised pretraining, the improvement may diminish with richer features or more balanced data splits. Thirdly, hyper-parameters could have larger impacts on accuracy of downstream tasks than the choice of pretraining tasks, especially when the scales of downstream tasks are small. Finally, we provide our conjectures where the complexity of some pretraining methods on small molecules might be insufficient, followed by empirical evidences on different pretraining datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑