arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.08910cs.LGcs.IR
收藏

Sharpness-Aware Graph Collaborative Filtering(锐度感知图协同过滤)

Sharpness-Aware Graph Collaborative Filtering

Visa Research(维萨研究院)

Huiyuan Chen, Chin-Chia Michael Yeh, Yujie Fan, Yan Zheng, Junpeng Wang, Vivian Lai, Mahashweta Das, Hao Yang

AI总结:

针对图神经网络在协同过滤中因分布偏移、极小值选择不当导致泛化性差的问题,提出基于双层优化的gSAM训练方案,通过正则化损失景观平坦度提升模型泛化能力,实验验证了其优越性。

摘要:

图神经网络(GNNs)已在协同过滤任务中取得了出色的性能。然而,当训练数据与测试数据的分布对齐不佳时,GNNs往往会表现出较差的性能。此外,训练GNNs需要优化非凸神经网络,这类网络存在大量局部和全局极小值,它们在测试阶段的性能可能存在显著差异。因此,谨慎选择极小值至关重要。本文提出了一种有效的训练方案,名为gSAM(图锐度感知最小化),其核心原理是:更“平坦”的极小值比更“尖锐”的极小值具有更好的泛化能力。为实现这一目标,gSAM通过构建双层优化来正则化权重损失景观的平坦度:外层问题执行标准的模型训练,内层问题则帮助模型跳出尖锐的极小值。实验结果证明了我们提出的gSAM的优越性。

英文摘要:

Graph Neural Networks (GNNs) have achieved impressive performance in collaborative filtering. However, GNNs tend to yield inferior performance when the distributions of training and test data are not aligned well. Also, training GNNs requires optimizing non-convex neural networks with an abundance of local and global minima, which may differ widely in their performance at test time. Thus, it is essential to choose the minima carefully. Here we propose an effective training schema, called {gSAM}, under the principle that the \textit{flatter} minima has a better generalization ability than the \textit{sharper} ones. To achieve this goal, gSAM regularizes the flatness of the weight loss landscape by forming a bi-level optimization: the outer problem conducts the standard model training while the inner problem helps the model jump out of the sharp minima. Experimental results show the superiority of our gSAM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑