arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2308.08129cs.LGq-bio.QM
收藏

自监督预训练是否有利于分子性质预测中的外推?

Is Self-Supervised Pretraining Good for Extrapolation in Molecular Property Prediction?

The University of Tokyo(东京大学)

Shun Takashige, Masatoshi Hanai, Toyotaro Suzumura, Limin Wang, Kenjiro Taura

AI总结:

本文研究自监督预训练对分子性质预测外推能力的影响,提出实验框架并证明其虽不能准确预测绝对值,却可帮助模型学习未观测性质的相对趋势。

摘要:

材料性质预测在电池、半导体、催化剂和药物等多种应用的材料开发与发现中发挥着关键作用。近年来,人们越来越关注采用数据驱动方法,将机器学习技术与传统理论计算相结合。在材料科学中,对未观测值的预测通常被称为外推,这对于性质预测尤为关键,因为它使研究人员能够洞察可用数据范围之外的材料。然而,即使近期强大的机器学习模型取得了进展,准确外推仍被广泛认为是一个极具挑战性的问题。另一方面,自监督预训练是一种机器学习技术:模型首先使用相对简单的预训练任务在无标注数据上进行训练,然后再在有标注数据上针对目标任务进行训练。由于自监督预训练能够有效利用没有观测性质值的材料数据,因此它有可能提升模型的外推能力。在本文中,我们阐明了这种自监督预训练如何增强外推性能。我们提出了一个用于演示的实验框架,并通过实证揭示:尽管模型无法准确外推绝对性质值,但自监督预训练使它们能够学习未观测性质值的相对趋势,并改善外推性能。

英文摘要:

The prediction of material properties plays a crucial role in the development and discovery of materials in diverse applications, such as batteries, semiconductors, catalysts, and pharmaceuticals. Recently, there has been a growing interest in employing data-driven approaches by using machine learning technologies, in combination with conventional theoretical calculations. In material science, the prediction of unobserved values, commonly referred to as extrapolation, is particularly critical for property prediction as it enables researchers to gain insight into materials beyond the limits of available data. However, even with the recent advancements in powerful machine learning models, accurate extrapolation is still widely recognized as a significantly challenging problem. On the other hand, self-supervised pretraining is a machine learning technique where a model is first trained on unlabeled data using relatively simple pretext tasks before being trained on labeled data for target tasks. As self-supervised pretraining can effectively utilize material data without observed property values, it has the potential to improve the model's extrapolation ability. In this paper, we clarify how such self-supervised pretraining can enhance extrapolation performance.We propose an experimental framework for the demonstration and empirically reveal that while models were unable to accurately extrapolate absolute property values, self-supervised pretraining enables them to learn relative tendencies of unobserved property values and improve extrapolation performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑