通过鲁棒直接偏好优化与稀疏MoE对齐多模态序列推荐
Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
AI总结：
本文研究了在隐式反馈下直接偏好优化的行为，通过系统实验比较了常见负样本选择策略及其与DPO训练的交互，发现用动态top-K候选池进行随机采样可提升排名性能，原因在于减少错误抑制梯度和保留信息硬信号。
摘要：
基于偏好对齐的目标已广泛应用于大语言模型中的RLHF式成对学习以及推荐系统中的新兴应用。然而，现有工作很少探讨直接偏好优化（DPO）在隐式反馈下的表现，其中未观察到的项目不可靠。我们对多模态序列推荐进行了系统实验，比较了常见负样本选择策略及其与DPO训练的交互。我们的核心发现是，简单修改，即用动态top-K候选池进行随机采样代替确定性硬负样本，能一致提高排名性能。我们将其有效性归因于两个因素：（1）减少由假负样本引起的错误抑制梯度；（2）保留信息硬信号并通过受控的随机性平滑优化。通过可选的稀疏混合专家编码器实现高效容量扩展，RoDPO在三个Amazon基准上达到高达5.25%的NDCG@5，推理成本几乎不变。
英文摘要：
Preference-based alignment objectives have been widely adopted, from RLHF-style pairwise learning in large language models to emerging applications in recommender systems. Yet, existing work rarely examines how Direct Preference Optimization (DPO) behaves under implicit feedback, where unobserved items are not reliable negatives. We conduct systematic experiments on multimodal sequential recommendation to compare common negative-selection strategies and their interaction with DPO training. Our central finding is that a simple modification, replacing deterministic hard negatives with stochastic sampling from a dynamic top-K candidate pool, consistently improves ranking performance. We attribute its effectiveness to two factors: (1) reducing erroneous suppressive gradients caused by false negatives, and (2) retaining informative hard signals while smoothing optimization via controlled stochasticity. With an optional sparse Mixture-of-Experts encoder for efficient capacity scaling, RoDPO achieves up to 5.25% NDCG@5 on three Amazon benchmarks, with nearly unchanged inference cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
