arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.13627cs.LGcs.CL
收藏

分子属性预测的BERT学

BERTology of Molecular Property Prediction

Department of Chemistry, Virginia Tech(弗吉尼亚理工大学化学系);Molecular Sciences Software Institute(分子科学软件研究所)

Mohammad Mostafanejad, Paul Saxe, T. Daniel Crawford

AI总结:

研究通过系统实验分析影响化学语言模型在分子属性预测任务中的预训练和微调性能的因素,揭示CLMs在不同基准任务中的表现差异及潜在机制。

摘要:

化学语言模型(CLMs)已涌现出作为分子属性预测(MPP)任务中流行经典机器学习模型的有力竞争者。然而,越来越多的研究报告了CLMs在各种MPP基准任务中性能不一致和矛盾的结果。在本研究中,我们进行了并分析了数百个精心控制的实验,系统地探讨了数据集大小、模型大小和标准化等因素对CLMs在MPP任务中的预训练和微调性能的影响。在缺乏针对仅编码器掩码语言模型的已建立的缩放定律的情况下,我们的目标是提供全面的数值证据和对影响CLMs在MPP任务中性能的潜在机制的深入理解,其中一些似乎在文献中被完全忽视。

英文摘要:

Chemical language models (CLMs) have emerged as promising competitors to popular classical machine learning models for molecular property prediction (MPP) tasks. However, an increasing number of studies have reported inconsistent and contradictory results for the performance of CLMs across various MPP benchmark tasks. In this study, we conduct and analyze hundreds of meticulously controlled experiments to systematically investigate the effects of various factors, such as dataset size, model size, and standardization, on the pre-training and fine-tuning performance of CLMs for MPP. In the absence of well-established scaling laws for encoder-only masked language models, our aim is to provide comprehensive numerical evidence and a deeper understanding of the underlying mechanisms affecting the performance of CLMs for MPP tasks, some of which appear to be entirely overlooked in the literature.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑