arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.12384cs.IRcs.LG
收藏

AlignRec:多模态推荐中的对齐与训练

AlignRec: Aligning and Training in Multimodal Recommendations

Shanghai Jiao Tong University(上海交通大学);Xiaohongshu Inc.(小红书公司)

Yifan Liu, Kangning Zhang, Xiangyuan Ren, Yanhua Huang, Jiarui Jin, Yingjie Qin, Ruilong Su, Ruiwen Xu, Yong Yu, Weinan Zhang

AI总结:

本文针对多模态推荐中内容特征与ID特征不对齐的问题,提出AlignRec框架,将推荐目标分解为内容内部、内容与ID、用户与物品三个对齐,并通过预训练统一多模态特征和设计新评估指标,在三个数据集上验证了其优于九个基线方法。

摘要:

随着多媒体系统的发展,多模态推荐正发挥着至关重要的作用,因为它们能够利用交互之外的丰富上下文信息。现有方法主要将多模态信息视为辅助信息,利用它们帮助学习ID特征;然而,多模态内容特征与基于ID的特征之间存在语义鸿沟,直接使用多模态信息作为辅助会导致用户和物品表示的不对齐。在本文中,我们首先系统地研究了多模态推荐中的不对齐问题,并提出了一种名为AlignRec的解决方案。在AlignRec中,推荐目标被分解为三个对齐,即内容内部的对齐、内容与类别ID之间的对齐,以及用户与物品之间的对齐。每个对齐都由一个特定的目标函数来刻画,并被集成到我们的多模态推荐框架中。为了有效地训练AlignRec,我们提出从预训练第一个对齐开始,以获得统一的多模态特征,随后以这些特征作为输入,共同训练接下来的两个对齐。由于分析每个多模态特征是否有助于训练并加速推荐模型的迭代周期至关重要,我们设计了三类新的指标来评估中间性能。我们在三个真实世界数据集上的大量实验一致验证了AlignRec相较于九个基线的优越性。我们还发现,AlignRec生成的多模态特征优于当前使用的特征,这些特征将在我们的仓库https://github.com/sjtulyf123/AlignRec_CIKM24中开源。

英文摘要:

With the development of multimedia systems, multimodal recommendations are playing an essential role, as they can leverage rich contexts beyond interactions. Existing methods mainly regard multimodal information as an auxiliary, using them to help learn ID features; However, there exist semantic gaps among multimodal content features and ID-based features, for which directly using multimodal information as an auxiliary would lead to misalignment in representations of users and items. In this paper, we first systematically investigate the misalignment issue in multimodal recommendations, and propose a solution named AlignRec. In AlignRec, the recommendation objective is decomposed into three alignments, namely alignment within contents, alignment between content and categorical ID, and alignment between users and items. Each alignment is characterized by a specific objective function and is integrated into our multimodal recommendation framework. To effectively train AlignRec, we propose starting from pre-training the first alignment to obtain unified multimodal features and subsequently training the following two alignments together with these features as input. As it is essential to analyze whether each multimodal feature helps in training and accelerate the iteration cycle of recommendation models, we design three new classes of metrics to evaluate intermediate performance. Our extensive experiments on three real-world datasets consistently verify the superiority of AlignRec compared to nine baselines. We also find that the multimodal features generated by AlignRec are better than currently used ones, which are to be open-sourced in our repository https://github.com/sjtulyf123/AlignRec_CIKM24.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑