arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2208.01684cs.LGcs.AIstat.ML
收藏

基于曲率的图网络多任务学习

Curvature-informed multi-task learning for graph networks

Johns Hopkins University Applied Physics Laboratory(约翰斯·霍普金斯大学应用物理实验室)

Alexander New, Michael J. Pekala, Nam Q. Le, Janna Domenico, Christine D. Piatko, Christopher D. Stiles

摘要:

晶体和分子的感兴趣的性质,如带隙,弹性和溶解度,通常彼此相关:它们受相同的物理定律支配。然而,当最先进的图神经网络试图同时预测多个属性(多任务学习(MTL)设置)时,它们经常表现不佳。这表明图形网络可能没有充分利用这些潜在的相似性。在这里,我们研究了这种现象的一个潜在解释:每个属性的损失曲面的曲率变化很大,导致学习效率低下。这种曲率差异可以通过查看每个属性的损失函数的Hessian的谱属性来评估,这是通过随机数值线性代数以无矩阵的方式完成的。我们在两个基准数据集(材料项目(MP)和QM8)上评估了我们的假设,并考虑这些发现如何为新型多任务学习模型的训练提供信息。

英文摘要:

Properties of interest for crystals and molecules, such as band gap, elasticity, and solubility, are generally related to each other: they are governed by the same underlying laws of physics. However, when state-of-the-art graph neural networks attempt to predict multiple properties simultaneously (the multi-task learning (MTL) setting), they frequently underperform a suite of single property predictors. This suggests graph networks may not be fully leveraging these underlying similarities. Here we investigate a potential explanation for this phenomenon: the curvature of each property's loss surface significantly varies, leading to inefficient learning. This difference in curvature can be assessed by looking at spectral properties of the Hessians of each property's loss function, which is done in a matrix-free manner via randomized numerical linear algebra. We evaluate our hypothesis on two benchmark datasets (Materials Project (MP) and QM8) and consider how these findings can inform the training of novel multi-task learning models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑