化学预训练模型系统综述
A Systematic Survey of Chemical Pre-trained Models
AI总结:
本文综述分子预训练模型 CPMs,从分子描述符、编码器架构、预训练策略与应用等方面系统梳理进展,并指出挑战与未来方向。
摘要:
深度学习在学习分子表示方面取得了显著成功,这对于从性质预测到药物设计等各种生化应用至关重要。然而,从头训练深度神经网络(DNNs)通常需要大量带标签分子,而这些分子在现实世界中获取成本高昂。为缓解这一问题,研究者投入了大量精力研究分子预训练模型(CPMs):先利用大规模无标签分子数据库预训练 DNNs,再在特定下游任务上进行微调。尽管该领域蓬勃发展,却仍缺乏对这一快速发展方向的系统性综述。本文提出首个总结 CPMs 当前进展的综述。我们首先强调从头训练分子表示模型的局限性,以此说明 CPM 研究的动机;随后从分子描述符、编码器架构、预训练策略和应用等多个关键视角,系统梳理该主题的最新进展。我们还重点指出未来研究面临的挑战与有前景的方向,为机器学习和科学社区提供有用资源。
英文摘要:
Deep learning has achieved remarkable success in learning representations for molecules, which is crucial for various biochemical applications, ranging from property prediction to drug design. However, training Deep Neural Networks (DNNs) from scratch often requires abundant labeled molecules, which are expensive to acquire in the real world. To alleviate this issue, tremendous efforts have been devoted to Molecular Pre-trained Models (CPMs), where DNNs are pre-trained using large-scale unlabeled molecular databases and then fine-tuned over specific downstream tasks. Despite the prosperity, there lacks a systematic review of this fast-growing field. In this paper, we present the first survey that summarizes the current progress of CPMs. We first highlight the limitations of training molecular representation models from scratch to motivate CPM studies. Next, we systematically review recent advances on this topic from several key perspectives, including molecular descriptors, encoder architectures, pre-training strategies, and applications. We also highlight the challenges and promising avenues for future research, providing a useful resource for both machine learning and scientific communities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。