arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2201.02327cs.IR
收藏

关于采样Softmax损失在物品推荐中的有效性研究

On the Effectiveness of Sampled Softmax Loss for Item Recommendation

Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, Tianyu Qiu

AI总结:

本研究针对推荐领域对采样Softmax(SSM)损失研究不足的问题,理论分析了SSM的三大优势及余弦相似度的模长学习缺陷,证明图卷积网络的消息传递可弥补该缺陷,实验验证了SSM在物品推荐中的优越性。

摘要:

学习目标在构建推荐系统中起着基础性作用。大多数方法通常采用逐点或成对损失来训练模型参数,而由于softmax损失在扩展到大型数据集时计算复杂度高,或是对于流式数据难以处理,因此很少受到关注。采样softmax(SSM)损失作为softmax损失的高效替代方案应运而生。它的特例InfoNCE损失已被广泛应用于自监督学习,并在对比学习中展现出卓越性能。然而,将SSM损失作为学习目标的推荐研究十分有限。更糟糕的是,据我们所知,现有研究均未深入探究其特性,也未能解答“SSM损失是否适用于物品推荐?”以及“与主流损失相比,SSM损失在概念上有哪些优势?”这两个问题。\n在本研究中,我们旨在更好地理解SSM在物品推荐中的作用。具体而言,我们首先从理论上揭示了三个与模型无关的优势:(1)缓解流行度偏差;(2)挖掘难负样本;(3)最大化排序指标。然而,基于我们的实证研究,我们发现SSM中默认选择的余弦相似度函数限制了其学习表示向量模长的能力。因此,将SSM与同样无法调整模长的模型结合,可能会导致表示效果不佳。进一步地,我们提供了数学证明,表明图卷积网络中的消息传递机制可以根据节点度调整表示模长,这自然弥补了SSM的不足。在四个基准数据集上的大量实验验证了我们的分析,证明了SSM在物品推荐中的优越性。我们的实现同时提供了TensorFlow和PyTorch版本。

英文摘要:

The learning objective plays a fundamental role to build a recommender system. Most methods routinely adopt either pointwise or pairwise loss to train the model parameters, while rarely pay attention to softmax loss due to its computational complexity when scaling up to large datasets or intractability for streaming data. The sampled softmax (SSM) loss emerges as an efficient substitute for softmax loss. Its special case, InfoNCE loss, has been widely used in self-supervised learning and exhibited remarkable performance for contrastive learning. Nonetheless, limited recommendation work uses the SSM loss as the learning objective. Worse still, none of them explores its properties thoroughly and answers ``Does SSM loss suit for item recommendation?'' and ``What are the conceptual advantages of SSM loss, as compared with the prevalent losses?'', to the best of our knowledge. In this work, we aim to offer a better understanding of SSM for item recommendation. Specifically, we first theoretically reveal three model-agnostic advantages: (1) mitigating popularity bias; (2) mining hard negative samples; and (3) maximizing the ranking metric. However, based on our empirical studies, we recognize that the default choice of cosine similarity function in SSM limits its ability in learning the magnitudes of representation vectors. As such, the combinations of SSM with the models that also fall short in adjusting magnitudes may result in poor representations. One step further, we provide mathematical proof that message passing schemes in graph convolution networks can adjust representation magnitude according to node degree, which naturally compensates for the shortcoming of SSM. Extensive experiments on four benchmark datasets justify our analyses, demonstrating the superiority of SSM for item recommendation. Our implementations are available in both TensorFlow and PyTorch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑