学习多步检索个人情境以实现个性化问答
Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering
AI总结：
PR2通过强化学习框架结合推理与检索，提升个性化问答的准确性和用户特定偏好对齐度。
摘要：
个性化问答（QA）需要既准确又与用户背景、偏好和历史情境一致的答案。现有最先进的方法主要依赖于检索增强生成（RAG）解决方案，通过从用户档案中检索相关项目来构建个人情境。现有方法直接使用用户的查询来检索个人文档，此类策略往往导致表面层面的个性化。我们提出PR2（个性化检索增强推理），一种结合从个人情境中推理和检索的强化学习框架，用于个性化。PR2学习适应性的检索-推理策略，确定何时检索、从用户档案中检索什么证据，以及如何将其纳入中间推理步骤。通过在个性化奖励函数下优化多轮推理轨迹，该框架强化了更符合用户特定偏好和通过奖励模型反映的上下文信号的推理路径。在使用三个LLM的LaMP-QA基准上的大量实验表明，PR2在个性化QA中始终优于强大的基线，实现了平均相对改进8.8%-12%。
英文摘要：
Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users' background, preferences, and historical context. Existing state-of-the-art methods primarily rely on retrieval-augmented generation (RAG) solutions that construct personal context by retrieving relevant items from the user's profile. Existing methods use the user's query directly to retrieve personal documents, and such strategies often lead to surface-level personalization. We propose PR2 (Personalized Retrieval-Augmented Reasoning), a reinforcement learning framework that integrates reasoning and retrieval from personal context for personalization. PR2 learns adaptive retrieval-reasoning policies, determining when to retrieve, what evidence to retrieve from user profiles, and how to incorporate it into intermediate reasoning steps. By optimizing multi-turn reasoning trajectories under a personalized reward function, the framework reinforces reasoning paths that better align with user-specific preferences and contextual signals reflected by the reward model. Extensive experiments on the LaMP-QA benchmark using three LLMs show that PR2 consistently outperforms strong baselines, achieving an average relative improvement of 8.8%-12% in personalized QA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
