arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00682cs.IRcs.AI
收藏

RecGOAT: 用于LLM增强多模态推荐的图最优自适应传输与双语义对齐

RecGOAT: Graph Optimal Adaptive Transport for LLM-Enhanced Multimodal Recommendation with Dual Semantic Alignment

Fudan University(复旦大学);University of Southern California(南加州大学)

Yuecheng Li, Hengwei Ju, Zeyu Song, Wei Yang, Chi Lu, Peng Jiang, Kun Gai

AI总结:

针对生成式语言模型表示与ID协同信号之间的语义异质性,提出基于图神经网络和最优传输理论的双粒度语义对齐框架RecGOAT,通过实例级和分布级对齐实现统一特征空间,理论证明其表示误差更低,实验达到最优性能。

摘要:

将大型语言模型(LLM)表示整合到多模态推荐中已显示出前景,但一个基本挑战仍被忽视:生成式LM表示与推荐系统依赖的基于ID的协同信号之间的语义异质性。未经对齐而简单注入LM特征会降低推荐性能而非提升。为解决此问题,我们提出RecGOAT,一个基于图神经网络和最优传输理论的双粒度语义对齐框架。RecGOAT首先通过多模态注意力图丰富协同语义,捕获物品-物品、用户-物品和用户-用户关系,并通过LLM推断的行为偏好初始化用户表示。然后,它在两个互补粒度上对齐LM导出的模态表示与推荐ID:(1)通过跨模态对比学习(CMCL)进行实例级对齐,产生判别性的每个样本表示;(2)通过最优自适应传输(OAT)进行分布级对齐,最小化ID分布与LLM语义之间的1-Wasserstein距离,以产生统一、一致对齐的特征空间。理论上,我们证明统一表示比任何单模态表示具有严格更低的目标误差,该误差界由Wasserstein距离和InfoNCE损失界定,为对齐一致性和融合全面性提供了严格保证。在三个公开基准上的大量实验展示了最先进的性能。在大规模在线广告平台上的部署进一步验证了RecGOAT的工业可扩展性。我们的代码可在https://github.com/6lyc/RecGOAT-LLM4Rec获取。

英文摘要:

Integrating large language model (LLM) representations into multimodal recommendation has shown promise, yet a fundamental challenge remains largely overlooked: the semantic heterogeneity between generative LM representations and the ID-based collaborative signals that recommendation systems rely on. Naively injecting LM features without alignment degrades recommendation performance rather than improving it. To resolve this, we propose RecGOAT, a dual-granularity semantic alignment framework built on graph neural networks and optimal transport theory. RecGOAT first enriches collaborative semantics through multimodal attentive graphs that capture item-item, user-item, and user-user relationships, initializing user representations via LLM-inferred behavioral preferences. It then aligns LM-derived modality representations with recommendation IDs at two complementary granularities: (1) instance-level alignment via cross-modal contrastive learning (CMCL), which produces discriminative per-sample representations; and (2) distribution-level alignment via optimal adaptive transport (OAT), which minimizes the 1-Wasserstein distance between ID distributions and LLM semantics to produce a unified, consistently aligned feature space. Theoretically, we prove that the unified representation achieves strictly lower target error than any single-modality representation, with the gap bounded by the Wasserstein distance and the InfoNCE loss, providing rigorous guarantees for both alignment consistency and fusion comprehensiveness. Extensive experiments on three public benchmarks demonstrate state-of-the-art performance. Deployment on a large-scale online advertising platform further validates RecGOAT's industrial scalability. Our code is available at https://github.com/6lyc/RecGOAT-LLM4Rec.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑