arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.20109cs.IR
收藏

面向对话推荐系统的多目标内在奖励学习

Multi-Objective Intrinsic Reward Learning for Conversational Recommender Systems

Zhendong Chu, Nan Wang, Hongning Wang

AI总结:

本文提出将内在奖励学习建模为多目标双层优化问题,以同时最大化对话推荐成功率和最小化对话轮数,并在三个公开CRS基准上验证了其显著提升推荐性能的有效性。

摘要:

对话推荐系统(CRS)主动获取用户偏好以生成自适应推荐。主流的基于强化学习的CRS解决方案严重依赖手工设计的奖励函数,这些奖励函数在CRS任务中可能无法与用户意图对齐。因此,设计任务特定的奖励对于促进CRS策略学习至关重要,而这一问题在现有文献中仍未得到充分探索。在本工作中,我们提出了一种新方法,通过从与用户的交互中学习内在奖励来应对这一挑战。具体而言,我们将内在奖励学习形式化为一个多目标双层优化问题。内层优化由学习到的内在奖励增强的CRS策略,而外层驱动内在奖励以优化两个CRS特定目标:最大化成功率,以及最小化在对话中达成成功推荐所需的轮数。为评估我们方法的有效性,我们在三个公开CRS基准上进行了大量实验。结果表明,我们的算法通过利用信息丰富的学习到的内在奖励,显著提升了CRS性能。

英文摘要:

Conversational Recommender Systems (CRS) actively elicit user preferences to generate adaptive recommendations. Mainstream reinforcement learning-based CRS solutions heavily rely on handcrafted reward functions, which may not be aligned with user intent in CRS tasks. Therefore, the design of task-specific rewards is critical to facilitate CRS policy learning, which remains largely under-explored in the literature. In this work, we propose a novel approach to address this challenge by learning intrinsic rewards from interactions with users. Specifically, we formulate intrinsic reward learning as a multi-objective bi-level optimization problem. The inner level optimizes the CRS policy augmented by the learned intrinsic rewards, while the outer level drives the intrinsic rewards to optimize two CRS-specific objectives: maximizing the success rate and minimizing the number of turns to reach a successful recommendation in conversations. To evaluate the effectiveness of our approach, we conduct extensive experiments on three public CRS benchmarks. The results show that our algorithm significantly improves CRS performance by exploiting informative learned intrinsic rewards.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑