arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.06286cs.IR
收藏

基于强化学习的推荐系统综述

Reinforcement learning based recommender systems: A survey

M. Mehdi Afsar, Trafford Crump, Behrouz Far

摘要:

推荐系统(RS)已经成为我们日常生活中不可分割的一部分。他们帮助我们找到我们最喜欢的项目购买,我们的朋友在社交网络,和我们最喜欢的电影看。传统上,推荐问题被认为是一个分类或预测问题,但它现在被广泛认为,制定它作为一个顺序决策问题,可以更好地反映用户-系统的交互。因此,它可以被公式化为马尔可夫决策过程(MDP),并解决了强化学习(RL)算法。与传统的推荐方法(包括协同过滤和基于内容的过滤)不同,RL能够处理连续的、动态的用户-系统交互,并考虑到长期的用户参与。虽然使用RL进行推荐的想法并不新鲜,并且已经存在了大约二十年,但它并不是很实用,主要是因为传统RL算法的可扩展性问题。然而,自从引入深度强化学习(DRL)以来,该领域出现了一个新的趋势,这使得将RL应用于具有大状态和动作空间的推荐问题成为可能。本文综述了基于强化学习的推荐系统。我们的目的是提出一个展望该领域,并为读者提供一个相当完整的知识领域的关键概念。我们首先认识到,并说明RLRS一般可以分为RL和DRL为基础的方法。然后,我们提出了一个RLRS框架,包括四个组件,即,状态表示、策略优化、奖励制定和环境构建,并相应地调查RLRS算法。我们强调新兴的主题,并使用各种图表描绘重要的趋势。最后,我们讨论了未来可以解决的重要方面和挑战。

英文摘要:

Recommender systems (RSs) have become an inseparable part of our everyday lives. They help us find our favorite items to purchase, our friends on social networks, and our favorite movies to watch. Traditionally, the recommendation problem was considered to be a classification or prediction problem, but it is now widely agreed that formulating it as a sequential decision problem can better reflect the user-system interaction. Therefore, it can be formulated as a Markov decision process (MDP) and be solved by reinforcement learning (RL) algorithms. Unlike traditional recommendation methods, including collaborative filtering and content-based filtering, RL is able to handle the sequential, dynamic user-system interaction and to take into account the long-term user engagement. Although the idea of using RL for recommendation is not new and has been around for about two decades, it was not very practical, mainly because of scalability problems of traditional RL algorithms. However, a new trend has emerged in the field since the introduction of deep reinforcement learning (DRL), which made it possible to apply RL to the recommendation problem with large state and action spaces. In this paper, a survey on reinforcement learning based recommender systems (RLRSs) is presented. Our aim is to present an outlook on the field and to provide the reader with a fairly complete knowledge of key concepts of the field. We first recognize and illustrate that RLRSs can be generally classified into RL- and DRL-based methods. Then, we propose an RLRS framework with four components, i.e., state representation, policy optimization, reward formulation, and environment building, and survey RLRS algorithms accordingly. We highlight emerging topics and depict important trends using various graphs and tables. Finally, we discuss important aspects and challenges that can be addressed in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑