用于动态推荐的最大熵正则化决策Transformer与奖励重标记
Maximum-Entropy Regularized Decision Transformer with Reward Relabelling for Dynamic Recommendation
AI总结:
本文提出EDT4Rec方法,通过最大熵增强探索策略和奖励重标记技术,解决了Decision Transformer在推荐中缺乏轨迹拼接能力和在线应用效果有限的问题,并在六个离线数据集和在线模拟器上验证了其有效性。
摘要:
基于强化学习的推荐系统近期获得了广泛关注。然而,由于仿真环境的典型局限性(如数据低效),大多数工作无法广泛应用于所有领域。为应对这些挑战,近期的进展利用了离线强化学习方法,以其利用离线数据集的数据驱动方法而著称。一个突出的例子是Decision Transformer。尽管它很受欢迎,但Decision Transformer方法存在固有缺陷,这在基于它的推荐方法中尤为明显。本文指出了现有基于Decision Transformer的方法的两个关键缺陷:缺乏拼接能力和在线应用效果有限。作为回应,我们引入了一种名为Max-Entropy enhanced Decision Transformer with Reward Relabeling for Offline RLRS(EDT4Rec)的新方法。我们的方法从最大熵视角出发,开发了最大熵增强的探索策略。该策略旨在促进在在线环境中进行更有效的探索。此外,为了增强模型拼接次优轨迹的能力,我们引入了一种独特的奖励重标记技术。为了验证EDT4Rec的有效性和优越性,我们在六个真实世界离线数据集和一个在线模拟器中进行了全面实验。
英文摘要:
Reinforcement learning-based recommender systems have recently gained popularity. However, due to the typical limitations of simulation environments (e.g., data inefficiency), most of the work cannot be broadly applied in all domains. To counter these challenges, recent advancements have leveraged offline reinforcement learning methods, notable for their data-driven approach utilizing offline datasets. A prominent example of this is the Decision Transformer. Despite its popularity, the Decision Transformer approach has inherent drawbacks, particularly evident in recommendation methods based on it. This paper identifies two key shortcomings in existing Decision Transformer-based methods: a lack of stitching capability and limited effectiveness in online adoption. In response, we introduce a novel methodology named Max-Entropy enhanced Decision Transformer with Reward Relabeling for Offline RLRS (EDT4Rec). Our approach begins with a max entropy perspective, leading to the development of a max entropy enhanced exploration strategy. This strategy is designed to facilitate more effective exploration in online environments. Additionally, to augment the model's capability to stitch sub-optimal trajectories, we incorporate a unique reward relabeling technique. To validate the effectiveness and superiority of EDT4Rec, we have conducted comprehensive experiments across six real-world offline datasets and in an online simulator.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。