基于等变Transformer的全面分子表征
Comprehensive Molecular Representation from Equivariant Transformer
AI总结:
针对分子模拟精度与性能的权衡问题,提出一种嵌入分子净电荷与自旋态的等变Transformer机器学习力场,在CH₂数据集上实现最优外推能力,还优化了激活函数、注意力温度并提出加速训练的权重初始化方法。
摘要:
如今,分子模拟中精度与性能之间的权衡问题可通过机器学习力场(MLFF)解决,该方法兼具从头算(ab initio)的精度与力场的数值效率。但与传统力场不同,将相关电子自由度纳入MLFF变得十分重要。本文实现了一种等变Transformer,无需额外神经网络参数即可嵌入分子净电荷与自旋态。在单重态/三重态非相关CH₂数据集上训练的模型能够识别不同自旋态,且展现出当前最优的外推能力。其中,自注意力机制可合理捕获非局域效应,我们证明该效应可通过网络超参数精细调节。我们确实发现,图网络自注意力机制中使用的Softmax激活函数在预测精度上优于类ReLU函数。将注意力温度从τ=√d提升至√(2d)可进一步改善外推能力,表明非局域性具有重要作用。此外,本文提出了一种权重初始化方法,可显著加速训练过程。
英文摘要:
The tradeoff between precision and performance in molecular simulations can nowadays be addressed by machine-learned force fields (MLFF), which combine \textit{ab initio} accuracy with force field numerical efficiency. Different from conventional force fields however, incorporating relevant electronic degrees of freedom into MLFFs becomes important. Here, we implement an equivariant transformer that embeds molecular net charge and spin state without additional neural network parameters. The model trained on a singlet/triplet non-correlated \ce{CH2} dataset can identify different spin states and shows state-of-the-art extrapolation capability. Therein, self-attention sensibly captures non-local effects, which, as we show, can be finely tuned over the network hyper-parameters. We indeed found that Softmax activation functions utilised in the self-attention mechanism of graph networks outperformed ReLU-like functions in prediction accuracy. Increasing the attention temperature from $τ= \sqrt{d}$ to $\sqrt{2d}$ further improved the extrapolation capability, indicating a weighty role of nonlocality. Additionally, a weight initialisation method was purposed that sensibly accelerated the training process.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。