面向多领域序列推荐的基于多预训练教师的课程调度知识蒸馏
Curriculum-scheduled Knowledge Distillation from Multiple Pre-trained Teachers for Multi-domain Sequential Recommendation
AI总结:
本文提出CKD-MDSR方法,通过课程调度从多个预训练教师模型蒸馏知识并选择性整合到学生模型中,解决了预训练推荐模型在多领域序列推荐中应用成本高的问题,在五个数据集上验证了有效性与高效性。
摘要:
预训练推荐模型(PRMs)近期受到越来越多的关注。然而,其内在异构的模型结构、庞大的模型规模和计算成本阻碍了它们在实际推荐系统中的采用。因此,探索如何在真实系统中高效地使用不同的预训练推荐模型至关重要。本文提出了一种新颖的面向多领域序列推荐的基于多预训练教师的课程调度知识蒸馏方法,称为CKD-MDSR,该方法充分利用不同的PRMs作为多个教师模型来增强一个小型学生推荐模型,并整合来自PRMs的多领域知识。具体而言,CKD-MDSR首先采用课程调度的用户行为序列采样,并从UniSRec和Recformer等代表性PRMs中联合蒸馏信息性知识。然后,在考虑置信度和一致性的前提下,将来自上述PRMs的知识选择性地整合到学生模型中。最后,我们在多领域序列推荐上验证了所提方法,并通过多种类型的学生模型(包括特征交互和基于图的推荐模型)进一步证明了其通用性。在五个真实数据集上的大量实验证明了CKD-MDSR的有效性和高效性,它可以被视为在真实系统中使用PRMs的高效捷径。
英文摘要:
Pre-trained recommendation models (PRMs) have received increasing interest recently. However, their intrinsically heterogeneous model structure, huge model size and computation cost hinder their adoptions in practical recommender systems. Hence, it is highly essential to explore how to use different pre-trained recommendation models efficiently in real-world systems. In this paper, we propose a novel curriculum-scheduled knowledge distillation from multiple pre-trained teachers for multi-domain sequential recommendation, called CKD-MDSR, which takes full advantages of different PRMs as multiple teacher models to boost a small student recommendation model, integrating the knowledge across multiple domains from PRMs. Specifically, CKD-MDSR first adopts curriculum-scheduled user behavior sequence sampling and distills informative knowledge jointly from the representative PRMs such as UniSRec and Recformer. Then, the knowledge from the above PRMs are selectively integrated into the student model in consideration of their confidence and consistency. Finally, we verify the proposed method on multi-domain sequential recommendation and further demonstrate its universality with multiple types of student models, including feature interaction and graph based recommendation models. Extensive experiments on five real-world datasets demonstrate the effectiveness and efficiency of CKD-MDSR, which can be viewed as an efficient shortcut using PRMs in real-world systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。