发表机构
Saarland University（萨尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
通过探测78种分子子结构，系统研究了预训练和微调对化学语言模型分子结构感知能力的影响，发现预训练提升上层结构意识，微调增强任务相关子结构表征。
AI 中文摘要
化学语言模型（CLMs）使用线性化表示（如SMILES）进行训练，但目前尚不清楚它们编码了哪些具有化学意义的子结构。为了增进对CLMs的理解，我们进行了一项系统性研究，探测了8个预训练模型和6个随机初始化模型中的78种分子子结构。此外，我们还研究了化学下游任务的微调如何影响分子子结构的学习表征。我们的结果表明，预训练通常提高了CLMs的分子结构意识，尤其是在上层。此外，随机初始化模型在第一层已经很好地编码了环结构。我们对两个化学下游任务的分析进一步揭示，有趣的是，微调对任务相关分子子结构的影响大于其他子结构，表明表征的变化遵循化学理论。
英文摘要
Chemical language models (CLMs) are trained with linearized representations such as SMILES, yet it remains unclear which chemically meaningful substructures they encode. To foster a better understanding of CLMs, we conduct a systematic study and probe for 78 molecular substructures across eight pre-trained and six randomly initialized models. We furthermore study how fine-tuning on chemical downstream tasks affects the learned representations of molecular substructures. Our results show that pre-training generally improves molecular structure awareness of CLMs, particularly in the upper layers. Moreover, randomly initialized models already encode ring structures well in the first layer. Our analysis on two chemical downstream tasks further reveals that, interestingly, fine-tuning affects task-relevant molecular substructures more than others, indicating that the changes in the representations follow chemical theory.
CommentsAccepted at EMNLP 2026 (to appear)
