arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.18481cs.IRcs.AI
收藏

MDE:多模态推荐中的模态判别增强

MDE: Modality Discrimination Enhancement for Multi-modal Recommendation

Zhejiang University(浙江大学);Nanyang Technological University(南洋理工大学);Singapore University of Social Sciences(新加坡社会科学大学)

Hang Zhou, Yucheng Wang, Huijing Zhan

AI总结:

针对多模态推荐中模态特定特征被忽视的问题,提出MDE框架,通过新颖融合模块增强模态差异并引入节点级权衡机制,在三个公开数据集上显著优于现有方法。

摘要:

多模态推荐系统旨在通过将项目的内容特征与用户行为数据在不同模态间进行整合来提升性能。有效利用不同模态的特征需要解决两个挑战:保留跨模态的语义共性(模态共享)以及捕捉每个模态的独特特征(模态特定)。现有大多数方法侧重于对齐跨模态的特征空间,这有助于表示模态共享特征。然而,模态特定的差异往往被忽视,尤其是在模态之间存在显著语义差异时。为了解决这一问题,我们提出了模态独特性增强(MDE)框架,该框架优先提取模态特定信息以提高推荐准确性,同时保持共享特征。MDE通过一种新颖的多模态融合模块增强模态间的差异,并引入节点级权衡机制来平衡跨模态对齐与差异化。在三个公开数据集上进行的大量实验表明,我们的方法显著优于其他最先进的方法,证明了联合考虑模态共享和模态特定特征的有效性。

英文摘要:

Multi-modal recommendation systems aim to enhance performance by integrating an item's content features across various modalities with user behavior data. Effective utilization of features from different modalities requires addressing two challenges: preserving semantic commonality across modalities (modality-shared) and capturing unique characteristics for each modality (modality-specific). Most existing approaches focus on aligning feature spaces across modalities, which helps represent modality-shared features. However, modality-specific distinctions are often neglected, especially when there are significant semantic variations between modalities. To address this, we propose a Modality Distinctiveness Enhancement (MDE) framework that prioritizes extracting modality-specific information to improve recommendation accuracy while maintaining shared features. MDE enhances differences across modalities through a novel multi-modal fusion module and introduces a node-level trade-off mechanism to balance cross-modal alignment and differentiation. Extensive experiments on three public datasets show that our approach significantly outperforms other state-of-the-art methods, demonstrating the effectiveness of jointly considering modality-shared and modality-specific features.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑