MLLMRec-R1: 通过大型语言模型增强多模态序列推荐的推理能力
MLLMRec-R1: Incentivizing Reasoning Capability in Large Language Models for Multimodal Sequential Recommendation
AI总结：
MLLMRec-R1通过离线文本化视觉信号和混合粒度数据增强策略，提升多模态序列推荐中基于GRPO的推理效率与稳定性。
摘要：
组相对策略优化（GRPO）已成为提高大型语言模型（LLMs）推理和偏好对齐的后训练标准范式，并最近在基于LLM的推荐系统中展示了强大的有效性。然而，将基于GRPO的推理管道扩展到多模态序列推荐（MSR）中，使用多模态大型语言模型（MLLMs）面临根本性障碍。首先，MSR需要同时编码历史交互和多个候选物品的视觉内容，导致视觉标记主导输入，使基于组的展开成本随历史长度和候选集大小而增加，这使得基于GRPO的训练成本过高。其次，现有的思维链（CoT）监督在推荐场景中面临奖励膨胀问题，其中更高的训练奖励并不可靠地转化为更好的排序性能，可能导致捷径学习。为了解决这些挑战，我们提出了MLLMRec-R1，一种高效的、稳定的基于GRPO的推理框架，用于多模态序列推荐。MLLMRec-R1通过离线文本化视觉信号来消除昂贵的视觉标记，同时保持多模态语义，并通过细化和置信度意识评估构建高质量的多模态CoT监督。此外，一种混合粒度数据增强策略有选择地注入可靠的CoT样本，同时保留标准训练数据，缓解奖励膨胀并提高泛化稳定性。在三个基准数据集上的广泛实验表明，MLLMRec-R1在很大程度上优于最先进的方法，建立了用于多模态序列推荐的基于GRPO的推理管道。代码可在https://github.com/wangyu0627/MLLMRec-R1上获得。
英文摘要：
Group relative policy optimization (GRPO) has become a standard post-training paradigm for improving reasoning and preference alignment in large language models (LLMs), and has recently shown strong effectiveness in LLM-based recommender systems. However, extending GRPO-based reasoning pipelines to multimodal sequential recommendation (MSR) with multimodal large language models (MLLMs) faces fundamental obstacles. First, MSR requires jointly encoding visual content for both historical interactions and multiple candidate items, causing visual tokens to dominate the input and making the cost of group-based rollout scale with history length and candidate set size, which renders GRPO-based training prohibitively expensive. Second, existing Chain-of-Thought (CoT) supervision suffers from reward inflation in recommendation scenarios, where higher training rewards do not reliably translate into improved ranking performance and may induce shortcut learning. To address these challenges, we propose MLLMRec-R1, an efficient and stable GRPO-based reasoning framework for multimodal sequential recommendation. MLLMRec-R1 textualizes visual signals offline to eliminate expensive visual tokens while preserving multimodal semantics, and constructs high-quality multimodal CoT supervision through refinement and confidence-aware assessment. Furthermore, a mixed-grained data augmentation strategy selectively injects reliable CoT samples while retaining standard training data, mitigating reward inflation and improving generalization stability. Extensive experiments on three benchmark datasets demonstrate that MLLMRec-R1 consistently outperforms state-of-the-art methods, establishing a practical and effective GRPO-based reasoning pipeline for multimodal sequential recommendation. The code is available at https://github.com/wangyu0627/MLLMRec-R1.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
