arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.03769physics.chem-phcs.LGphysics.comp-ph
收藏

训练算法对神经网络潜力性能的影响:Adam和卡尔曼滤波器优化器的案例研究

Training Algorithm Matters for the Performance of Neural Network Potential: A Case Study of Adam and the Kalman Filter Optimizers

Uppsala University(乌普萨拉大学)

Yunqi Shao, Florian M. Dietrich, Carl Nettelblad, Chao Zhang

摘要:

开发神经网络潜力(NNPs)的一个隐藏但重要的问题是选择训练算法。在这里,我们比较了两种流行的训练算法,自适应矩估计算法(亚当)和扩展卡尔曼滤波算法(EKF)的性能,使用Behler-Parrinello神经网络(BPNN)和两个公开访问的数据集的液态水[Proc.Natl. Acad. Sci. U.S.A. 2016,113,8368-8373和Proc. Natl. Acad. Sci. U.S.A. 2019,116,1110-1115]。这是通过在TensorFlow中实现EKF来实现的。研究发现,与Adam相比,用EKF训练的NNPs更具可转移性,对学习率的值不太敏感。在这两种情况下,验证集的错误度量并不总是作为NNPs实际性能的良好指标。相反,我们表明,它们的性能相关性以及基于Fisher信息的相似性度量。

英文摘要:

One hidden yet important issue for developing neural network potentials (NNPs) is the choice of training algorithm. Here we compare the performance of two popular training algorithms, the adaptive moment estimation algorithm (Adam) and the Extended Kalman Filter algorithm (EKF), using the Behler-Parrinello neural network (BPNN) and two publicly accessible datasets of liquid water [Proc. Natl. Acad. Sci. U.S.A. 2016, 113, 8368-8373 and Proc. Natl. Acad. Sci. U.S.A. 2019, 116, 1110-1115]. This is achieved by implementing EKF in TensorFlow. It is found that NNPs trained with EKF are more transferable and less sensitive to the value of the learning rate, as compared to Adam. In both cases, error metrics of the validation set do not always serve as a good indicator for the actual performance of NNPs. Instead, we show that their performance correlates well with a Fisher information based similarity measure.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑