SlateFree：一种基于Slate行为的强化学习无模型分解
SlateFree: a Model-Free Decomposition for Reinforcement Learning with Slate Actions
摘要：
我们考虑的问题，顺序建议，在每一步代理提出一些石板$N$不同的项目给用户从一个更大的目录大小$K>>N$。用户对推荐有未知的偏好，代理在强化学习的帮助下采取顺序行动，优化（在我们的情况下最小化）一些与用户相关的成本。一个石板的可能的项目组合是$\binom{K}{N}$，一个巨大的数字渲染值迭代方法棘手。我们证明了石板MDP实际上可以分解使用每个状态，它描述了一个更紧凑，更有效的方式问题的$K$项目相关的$Q$功能。在此基础上，我们提出了一种新的无模型SARSA和Q-学习算法，执行$N$并行迭代每一步，没有任何事先用户的知识。我们称这种方法为\texttt{SlateFree}，即免费的石板，我们数值计算表明，它收敛非常快，任意用户简档的确切最佳，它优于文献中的替代品。
英文摘要：
We consider the problem of sequential recommendations, where at each step an agent proposes some slate of $N$ distinct items to a user from a much larger catalog of size $K>>N$. The user has unknown preferences towards the recommendations and the agent takes sequential actions that optimise (in our case minimise) some user-related cost, with the help of Reinforcement Learning. The possible item combinations for a slate is $\binom{K}{N}$, an enormous number rendering value iteration methods intractable. We prove that the slate-MDP can actually be decomposed using just $K$ item-related $Q$ functions per state, which describe the problem in a more compact and efficient way. Based on this, we propose a novel model-free SARSA and Q-learning algorithm that performs $N$ parallel iterations per step, without any prior user knowledge. We call this method \texttt{SlateFree}, i.e. free-of-slates, and we show numerically that it converges very fast to the exact optimum for arbitrary user profiles, and that it outperforms alternatives from the literature.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
