屏蔽的推荐强化学习：无需降级的推荐系统解释生成
Shielded RecRL: Explanation Generation for Recommender Systems without Ranking Degradation
AI总结：
Shielded RecRL通过双塔架构和复合奖励信号，在不降低推荐准确性的情况下，提升推荐系统个性化解释的质量和用户交互效果。
摘要：
我们介绍了Shielded RecRL，一种强化学习方法，用于生成推荐系统中的个性化解释，而不会牺牲系统的原始排序性能。与以往基于RLHF的推荐方法不同，我们的双塔架构保持推荐系统的排序模型完整，同时语言模型学习生成有用的解释。我们设计了一个复合奖励信号，结合解释长度、内容相关性和连贯性，并应用具有KL散度约束的近端策略优化（PPO）来微调大型语言模型，仅使用LoRA适配器的0.4%参数进行训练。在Amazon Books数据集（约5万次交互在奇幻和浪漫类别中）的实验中，Shielded RecRL将相对点击通过率（CTR）提高了22.5%（1.225倍于基线），同时保持推荐系统的物品排序行为基本不变。广泛的消融研究证实，我们的梯度屏蔽策略和奖励设计有效平衡了解释质量和策略漂移。我们的结果表明，通过丰富的个性化解释，Shielded RecRL增强了面向用户的推荐方面，而不会降低核心推荐准确性。
英文摘要：
We introduce Shielded RecRL, a reinforcement learning approach to generate personalized explanations for recommender systems without sacrificing the system's original ranking performance. Unlike prior RLHF-based recommender methods that directly optimize item rankings, our two-tower architecture keeps the recommender's ranking model intact while a language model learns to produce helpful explanations. We design a composite reward signal combining explanation length, content relevance, and coherence, and apply proximal policy optimization (PPO) with a KL-divergence constraint to fine-tune a large language model with only 0.4% of its parameters trainable via LoRA adapters. In experiments on an Amazon Books dataset (approximately 50K interactions in the fantasy and romance genres), Shielded RecRL improved the relative click-through rate (CTR) by 22.5% (1.225x over baseline) while keeping the recommender's item-ranking behavior virtually unchanged. An extensive ablation study confirms that our gradient shielding strategy and reward design effectively balance explanation quality and policy drift. Our results demonstrate that Shielded RecRL enhances user-facing aspects of recommendations through rich, personalized explanations without degrading core recommendation accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
