利用好奇心奖励增强个性化多轮对话
Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward
AI总结：
针对现有多轮对话个性化依赖用户历史、传统RLHF共情适配不足的问题，提出融入好奇心内在奖励的多轮RLHF方法，可主动推断用户特征，在推荐与教育场景提升个性化效果与泛化能力。
摘要：
大语言模型（LLM）等有效对话智能体必须实现交互个性化，以适配教育、医疗等多元领域中用户的偏好、性格与属性。当前基于人类反馈的强化学习（RLHF，Reinforcement Learning from Human Feedback）等方法通常优先考量有用性与安全性，但在打造真正共情、自适应且个性化的对话方面存在不足。现有个性化方法多依赖大量用户历史数据，对新用户或上下文有限的用户效果受限。\n为解决这些局限，我们提出借助用户模型，将基于好奇心的内在奖励融入多轮RLHF。这种新型奖励机制通过优化对话以提升用户模型的准确率，激励LLM智能体主动推断用户特征，进而通过加深对用户的了解提供更个性化的交互。\n我们在两个不同领域验证了方法的有效性：在对话推荐任务中显著提升个性化性能，以及在教育场景中针对不同学习风格实现对话个性化。结果表明，与传统多轮RLHF相比，该方法泛化能力更强，且能维持对话质量，为打造更具个性化、自适应与吸引力的对话智能体提供了可行方案。
英文摘要：
Effective conversational agents like large language models (LLMs) must personalize their interactions to adapt to user preferences, personalities, and attributes across diverse domains like education and healthcare. Current methods like Reinforcement Learning from Human Feedback (RLHF), often prioritize helpfulness and safety but fall short in fostering truly empathetic, adaptive, and personalized dialogues. Existing personalization approaches typically rely on extensive user history, limiting their effectiveness for new or context-limited users. To address these limitations, we propose leveraging a user model to incorporate a curiosity-based intrinsic reward into multi-turn RLHF. This novel reward mechanism encourages the LLM agent to actively infer user traits by optimizing conversations to improve its user model's accuracy. Consequently, the agent delivers more personalized interactions by learning more about the user. We demonstrate our method's effectiveness in two distinct domains: significantly improving personalization performance in a conversational recommendation task, and personalizing conversations for different learning styles in an educational setting. We show improved generalization capabilities compared to traditional multi-turn RLHF, all while maintaining conversation quality. Our method offers a promising solution for creating more personalized, adaptive, and engaging conversational agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
