arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.07770cs.IR
收藏

推荐系统中负采样下损失函数的理论分析

A Theoretical Analysis of Recommendation Loss Functions under Negative Sampling

Giulia Di Teodoro, Federico Siciliano, Nicola Tonellotto, Fabrizio Silvestri

AI总结:

本文理论分析了推荐系统中CCE、BCE和BPR三种损失函数在负采样下的下界性质,证明CCE提供最紧下界,且单负样本时BPR与CCE等价,实验验证了理论结果。

摘要:

分类交叉熵(CCE)、二元交叉熵(BCE)和贝叶斯个性化排序(BPR)等损失函数常用于训练推荐系统(RSs),以区分正样本(用户交互过的物品)和负样本。先前的工作通过实验表明,在使用全部负样本时,CCE优于BCE和BPR,我们通过证明CCE在归一化折损累积增益(NDCG)和平均倒数排名(MRR)等排名指标上提供了最紧的下界,其次是BPR和BCE,从而为此提供了理论解释。然而,对于大规模推荐系统,使用全部负样本在计算上不可行,因此促使了负采样技术的使用。在负采样下,我们揭示当抽取单个负样本时,BPR和CCE是等价的,并且所有三种损失收敛到相同的全局最小值。我们进一步证明,采样后的损失仍然是NDCG(MRR)的下界,尽管是在概率意义上。我们的最坏情况分析表明,BCE在NDCG(MRR)上提供了最强的界。在五个数据集和四个模型上的实验实证支持了这些理论发现。我们的代码和补充材料可在https://github.com/federicosiciliano/recsys_losses.git获取。

英文摘要:

Loss functions like Categorical Cross Entropy (CCE), Binary Cross Entropy (BCE), and Bayesian Personalized Ranking (BPR) are commonly used in training Recommender Systems (RSs) to differentiate positive items - those interacted with by users - and negative items. While prior works empirically showed that CCE outperforms BCE and BPR when using the full set of negative items, we provide a theoretical explanation for this by proving that CCE offers the tightest lower bound on ranking metrics like Normalized Discounted Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR), followed by BPR and BCE. However, using the full set of negative items is computationally infeasible for large-scale RSs, prompting the use of negative sampling techniques. Under negative sampling, we reveal that BPR and CCE are equivalent when a single negative sample is drawn, and all three losses converge to the same global minimum. We further demonstrate that the sampled losses remain lower bounds for NDCG (MRR), albeit in a probabilistic sense. Our worst-case analysis shows that BCE offers the strongest bound on NDCG (MRR). Experiments on five datasets and four models empirically support these theoretical findings. Our code and supplementary material are available at https://github.com/federicosiciliano/recsys_losses.git.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑