arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.04549cs.IR
收藏

面向推荐系统的偏好一致性知识蒸馏

Preference-Consistent Knowledge Distillation for Recommender System

Zhangchi Zhu, Wei Zhang

AI总结:

针对推荐系统特征知识蒸馏中存在的偏好不一致问题,提出含两个正则化项及混合策略的PCKD方法,聚焦高偏好物品缓解该问题,经多数据集和骨干模型验证有效并开源代码。

摘要:

基于特征的知识蒸馏已被用于压缩现代推荐模型,通常采用投影层将学生(小型)推荐模型的维度与教师模型维度对齐。然而,现有研究仅关注使投影后的特征(即经过投影层的学生特征)与教师特征相似,忽略了探究用户偏好能否通过这种方式迁移到学生特征(即投影层前的学生特征)中。本文发现,由于缺乏对投影层的约束,用户偏好的迁移过程很可能受到干扰,我们将这种现象称为偏好不一致,它极大地浪费了基于特征的知识蒸馏的能力。为缓解偏好不一致问题,我们提出了PCKD(偏好一致性知识蒸馏)方法,该方法包含两个针对投影层的正则化项,我们还提出了一种结合这两个正则化项的混合方法。我们聚焦于高偏好分数的物品,显著缓解了偏好不一致问题,提升了基于特征的知识蒸馏的性能。在三个公开数据集和三个骨干模型上开展的大量实验证明了PCKD的有效性。我们的方法代码已发布在https://github.com/woriazzc/KDs。

英文摘要:

Feature-based knowledge distillation has been applied to compress modern recommendation models, usually with projectors that align student (small) recommendation models' dimensions with teacher dimensions. However, existing studies have only focused on making the projected features (i.e., student features after projectors) similar to teacher features, overlooking investigating whether the user preference can be transferred to student features (i.e., student features before projectors) in this manner. In this paper, we find that due to the lack of restrictions on projectors, the process of transferring user preferences will likely be interfered with. We refer to this phenomenon as preference inconsistency. It greatly wastes the power of feature-based knowledge distillation. To mitigate preference inconsistency, we propose PCKD, which consists of two regularization terms for projectors. We also propose a hybrid method that combines the two regularization terms. We focus on items with high preference scores and significantly mitigate preference inconsistency, improving the performance of feature-based knowledge distillation. Extensive experiments on three public datasets and three backbones demonstrate the effectiveness of PCKD. The code of our method is provided in https://github.com/woriazzc/KDs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑