PrLM：通过对比奖励优化学习显式推理的个性化 RAG
PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization
AI总结：
针对个性化 RAG 中 LLM 隐式整合导致对检索质量敏感且易偏离用户偏好的问题，提出强化学习框架 PrLM，在对比训练的个性化奖励模型指导下显式推理用户画像，无需标注推理路径，在多个数据集上优于现有方法且保持稳健。
摘要：
个性化检索增强生成（RAG）旨在通过将检索到的用户画像与输入查询结合，生成符合用户定制需求的回复。现有方法主要侧重于改进检索，并依赖大语言模型（LLM）隐式地将检索到的上下文与查询整合。然而，此类模型通常对检索质量敏感，可能生成与用户偏好不一致的回复。为解决该局限，我们提出 PrLM，一种强化学习框架，训练 LLM 对检索到的用户画像进行显式推理。在经过对比训练的个性化奖励模型指导下，PrLM 能够有效地从用户回复中学习，而无需标注的推理路径。在三个个性化文本生成数据集上的实验表明，PrLM 优于现有方法，并且在不同的检索画像数量和不同检索器下保持稳健。
英文摘要：
Personalized retrieval-augmented generation (RAG) aims to produce user-tailored responses by incorporating retrieved user profiles alongside the input query. Existing methods primarily focus on improving retrieval and rely on large language models (LLMs) to implicitly integrate the retrieved context with the query. However, such models are often sensitive to retrieval quality and may generate responses that are misaligned with user preferences. To address this limitation, we propose PrLM, a reinforcement learning framework that trains LLMs to explicitly reason over retrieved user profiles. Guided by a contrastively trained personalization reward model, PrLM effectively learns from user responses without requiring annotated reasoning paths. Experiments on three personalized text generation datasets show that PrLM outperforms existing methods and remains robust across varying numbers of retrieved profiles and different retrievers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
