arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.09034cs.IRcs.AI
收藏

UOEP:面向用户探索策略以增强推荐系统中的长期用户体验

UOEP: User-Oriented Exploration Policy for Enhancing Long-Term User Experiences in Recommender Systems

Renmin University of China(中国人民大学);University of International Business and Economics(对外经济贸易大学)

Changshuo Zhang, Sirui Chen, Xiao Zhang, Sunhao Dai, Weijie Yu, Jun Xu

AI总结:

针对推荐系统中统一探索策略忽视用户活跃度差异的问题,提出用户导向探索策略(UOEP),通过分布性评论家和多样化行动者群体实现细粒度探索,实验证明其能提升长期性能、低活跃用户表现及公平性。

摘要:

强化学习(RL)通过有效探索用户兴趣,在增强推荐系统中的用户长期体验方面获得了越来越多的关注。然而,现代推荐系统在数千万个物品中展现出不同的用户行为模式,这增加了探索的难度。例如,不同活跃程度的用户行为需要不同强度的探索,而以往的研究往往忽视这一方面,对所有用户采用统一的探索策略,这最终会损害用户的长期体验。为了应对这些挑战,我们提出了用户导向探索策略(UOEP),一种促进用户群体间细粒度探索的新方法。我们首先构建了一个分布性评论家(distributional critic),它允许在用户累积奖励反馈的不同分位数水平下进行策略优化,代表不同活跃程度的用户群体。在该评论家的指导下,我们设计了一个由不同行动者(actors)组成的群体,旨在其各自的用户群体内进行有效且细粒度的探索。为了在探索过程中同时增强多样性和稳定性,我们进一步引入了群体级多样性正则化项和监督模块。在公开推荐数据集上的实验结果表明,我们的方法在长期性能方面优于所有其他基线,验证了其用户导向探索的有效性。同时,进一步的分析揭示了我们的方法在提高低活跃用户性能以及增加用户间公平性方面的优势。

英文摘要:

Reinforcement learning (RL) has gained traction for enhancing user long-term experiences in recommender systems by effectively exploring users' interests. However, modern recommender systems exhibit distinct user behavioral patterns among tens of millions of items, which increases the difficulty of exploration. For example, user behaviors with different activity levels require varying intensity of exploration, while previous studies often overlook this aspect and apply a uniform exploration strategy to all users, which ultimately hurts user experiences in the long run. To address these challenges, we propose User-Oriented Exploration Policy (UOEP), a novel approach facilitating fine-grained exploration among user groups. We first construct a distributional critic which allows policy optimization under varying quantile levels of cumulative reward feedbacks from users, representing user groups with varying activity levels. Guided by this critic, we devise a population of distinct actors aimed at effective and fine-grained exploration within its respective user group. To simultaneously enhance diversity and stability during the exploration process, we further introduce a population-level diversity regularization term and a supervision module. Experimental results on public recommendation datasets demonstrate that our approach outperforms all other baselines in terms of long-term performance, validating its user-oriented exploration effectiveness. Meanwhile, further analyses reveal our approach's benefits of improved performance for low-activity users as well as increased fairness among users.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑