低秩马尔可夫决策过程中可证明高效的表示选择：从在线到离线强化学习
Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL
AI总结：
研究低秩MDPs中的表示选择问题，提出ReLEX算法用于在线和离线RL，ReLEX-UCB在覆盖条件下实现严格更优的常数遗憾，ReLEX-LCB首次在离线RL表示学习中实现常数样本复杂度。
摘要：
深度强化学习（DRL）的成功在于其能够学习一种适合探索与利用任务的表示。为了理解表示的选择如何提升强化学习（RL）的效率，我们研究了一类低秩马尔可夫决策过程（MDPs）的表示选择问题，其中转移核可以表示为双线性形式。我们提出了一种名为ReLEX的高效算法，用于在线和离线RL中的表示学习。具体而言，我们证明了ReLEX的在线版本ReLEX-UCB，其表现始终不劣于无表示选择的现有最优算法，并且若表示函数类对整个状态-动作空间具有“覆盖”性质，则能实现严格更优的常数遗憾。对于离线对应的ReLEX-LCB，我们证明若表示类能覆盖状态-动作空间，该算法即可找到最优策略，并实现依赖于差距的样本复杂度。这是离线RL表示学习中首个具有常数样本复杂度的结果。
英文摘要：
The success of deep reinforcement learning (DRL) lies in its ability to learn a representation that is well-suited for the exploration and exploitation task. To understand how the choice of representation can improve the efficiency of reinforcement learning (RL), we study representation selection for a class of low-rank Markov Decision Processes (MDPs) where the transition kernel can be represented in a bilinear form. We propose an efficient algorithm, called ReLEX, for representation learning in both online and offline RL. Specifically, we show that the online version of ReLEX, called ReLEX-UCB, always performs no worse than the state-of-the-art algorithm without representation selection, and achieves a strictly better constant regret if the representation function class has a "coverage" property over the entire state-action space. For the offline counterpart, ReLEX-LCB, we show that the algorithm can find the optimal policy if the representation class can cover the state-action space and achieves gap-dependent sample complexity. This is the first result with constant sample complexity for representation learning in offline RL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
