arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.03032cs.IRcs.LG
收藏

用于结构感知推荐嵌入演化的图增强优化器

Graph-enhanced Optimizers for Structure-aware Recommendation Embedding Evolution

East China Normal University(华东师范大学);Tsinghua University(清华大学)

Cong Xu, Jun Wang, Jianyong Wang, Wei Zhang

AI总结:

提出结构感知嵌入演化机制SEvo,将图结构信息直接注入嵌入更新过程,可无缝集成至AdamW等优化器,在多模型和数据集上实现一致性能提升。

摘要:

嵌入在现代推荐系统中起着关键作用,因为它们是现实世界实体的虚拟表示,也是后续决策模型的基础。在本文中,我们提出了一种新颖的嵌入更新机制,即结构感知嵌入演化(简称SEvo),以鼓励相关节点在每一步中以相似的方式演化。与通常作为中间模块的GNN(图神经网络)不同,SEvo能够在训练期间以最小的计算开销将图结构信息直接注入到嵌入中。本文从理论上分析了SEvo及其潜在变体的收敛性质,以证明设计的有效性。此外,SEvo可以无缝集成到现有的优化器中,以实现最先进的性能。特别是,带有矩估计校正的SEvo增强型AdamW在一系列模型和数据集上表现出一致的改进,这表明了一条超越显式GNN模块来有效利用图结构信息的新颖技术路线。

英文摘要:

Embedding plays a key role in modern recommender systems because they are virtual representations of real-world entities and the foundation for subsequent decision-making models. In this paper, we propose a novel embedding update mechanism, Structure-aware Embedding Evolution (SEvo for short), to encourage related nodes to evolve similarly at each step. Unlike GNN (Graph Neural Network) that typically serves as an intermediate module, SEvo is able to directly inject graph structural information into embedding with minimal computational overhead during training. The convergence properties of SEvo along with its potential variants are theoretically analyzed to justify the validity of the designs. Moreover, SEvo can be seamlessly integrated into existing optimizers for state-of-the-art performance. Particularly SEvo-enhanced AdamW with moment estimate correction demonstrates consistent improvements across a spectrum of models and datasets, suggesting a novel technical route to effectively utilize graph structural information beyond explicit GNN modules.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑