我们在群组推荐领域真的取得了进展吗？揭开平局打破的假象
Are We Really Making Progress in Group Recommendation? Unmasking the Tie-Breaking Illusion
浏览论文内容
中文总结 AI 辅助
本文揭示群组推荐领域存在训练时分数压缩与评估时确定性平局打破导致的评估偏差，经感知平局评估后，多数方法的改进大幅缩减，凸显该领域采用感知平局评估的必要性。
中文摘要 AI 辅助
近期的群组推荐方法在标准基准测试中取得了显著的性能提升，但这些增益是否始终反映了群组偏好建模的真正进步仍不明确。本文表明，若干近期方法存在系统性评估偏差，该偏差由训练时的分数压缩与评估时的确定性平局打破机制的相互作用导致。具体而言，在BPR损失前添加sigmoid变换可大幅增加平局的最高分数，使HR@K和NDCG@K等Top-K指标对平局的处理方式高度敏感。我们在CAMRa2011和Mafengwo数据集上，针对群组推荐和用户推荐两种设置，重新评估了近期代表性方法及其基线，并采用感知平局的评估协议，该协议计算了在均匀随机平局打破下HR@K和NDCG@K的精确期望。结果显示，在感知平局的评估下，许多先前报告的改进大幅缩减，方法间的相对排名也会发生显著变化。我们进一步表明，额外的sigmoid可能在优化过程中起到隐式边距平滑作用，而温度缩放的BPR可保留大部分此类优势，同时不会引发严重的平局膨胀。总体而言，我们的研究结果强调了感知平局评估对于确立群组推荐领域可靠进展的重要性。代码可在该https URL获取。
英文摘要
Recent group recommendation methods have reported strong improvements on standard benchmarks, but it remains unclear whether these gains always reflect genuine advances in modeling group preferences. In this paper, we show that several recent methods are affected by a systematic evaluation bias caused by the interaction between training-time score compression and evaluation-time deterministic tie-breaking. Specifically, an additional sigmoid transformation before the BPR objective can greatly increase tied top scores, making top-K metrics such as HR@K and NDCG@K highly sensitive to how ties are resolved. We revisit recent representative methods and their baselines on CAMRa2011 and Mafengwo under both group and user recommendation settings, and evaluate them with a tie-aware protocol that computes the exact expectation of HR@K and NDCG@K under uniform random tie-breaking. Our results show that many previously reported improvements shrink substantially under tie-aware evaluation, and the relative ranking of methods can change markedly. We further show that the additional sigmoid may act as implicit margin smoothing during optimization, and that temperature-scaled BPR can retain much of this benefit without inducing severe tie inflation. Overall, our findings highlight the importance of tie-aware evaluation for establishing reliable progress in group recommendation. The code is available at https://github.com/songduoma/TieAwareGroupRec.
