arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.03562cs.IR
收藏

面向推荐的以ID为中心的预训练

ID-centric Pre-training for Recommendation

Yiqing Wu, Ruobing Xie, Zhao Zhang, Fuzhen Zhuang, Xu Zhang, Leyu Lin, Zhanhui Kang, Yongjun Xu

AI总结:

本文提出以ID为中心的推荐预训练范式IDP,通过跨领域ID匹配器CDIM将预训练领域的行为性ID嵌入迁移到新领域物品表示,在冷启动和热启动场景中显著优于基线方法。

摘要:

经典序列推荐模型通常采用ID嵌入来存储从用户历史行为中学习到的知识并表征物品。然而,这些唯一ID难以迁移到新领域。随着预训练语言模型(PLM)的兴起,一些开创性工作采用PLM进行预训练推荐,其中模态信息(如文本)通过PLM被视为跨领域通用的信息。遗憾的是,在基于PLM的推荐模型中,ID嵌入中的行为信息仍被证实相比模态信息占据主导地位,从而限制了这些模型的性能。本文提出一种新颖的以ID为中心的推荐预训练范式(IDP),直接将预训练领域学习到的富含信息的ID嵌入迁移到新领域的物品表示中。具体而言,在预训练阶段,除了用于推荐的基于ID的序列模型外,我们还构建了一个由行为和模态信息共同学习的跨领域ID匹配器(CDIM)。在微调阶段,新领域物品的模态信息被视为由CDIM构建的跨领域桥梁。我们首先利用下游领域物品的文本信息,通过CDIM从预训练领域检索行为上和语义上相似的物品。随后,直接采用这些检索到的预训练ID嵌入(而非某些文本嵌入)来生成下游新物品的嵌入。通过在真实世界数据集上进行大量实验,在冷启动和热启动设置下,我们证明所提模型显著优于所有基线方法。代码将在录用后发布。

英文摘要:

Classical sequential recommendation models generally adopt ID embeddings to store knowledge learned from user historical behaviors and represent items. However, these unique IDs are challenging to be transferred to new domains. With the thriving of pre-trained language model (PLM), some pioneer works adopt PLM for pre-trained recommendation, where modality information (e.g., text) is considered universal across domains via PLM. Unfortunately, the behavioral information in ID embeddings is still verified to be dominating in PLM-based recommendation models compared to modality information and thus limits these models' performance. In this work, we propose a novel ID-centric recommendation pre-training paradigm (IDP), which directly transfers informative ID embeddings learned in pre-training domains to item representations in new domains. Specifically, in pre-training stage, besides the ID-based sequential model for recommendation, we also build a Cross-domain ID-matcher (CDIM) learned by both behavioral and modality information. In the tuning stage, modality information of new domain items is regarded as a cross-domain bridge built by CDIM. We first leverage the textual information of downstream domain items to retrieve behaviorally and semantically similar items from pre-training domains using CDIM. Next, these retrieved pre-trained ID embeddings, rather than certain textual embeddings, are directly adopted to generate downstream new items' embeddings. Through extensive experiments on real-world datasets, both in cold and warm settings, we demonstrate that our proposed model significantly outperforms all baselines. Codes will be released upon acceptance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑