arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2007.03749cs.LGstat.ML
收藏

平滑Bellman误差嵌入的精确分析

Sharp Analysis of Smoothed Bellman Error Embedding

Mila(米拉人工智能研究所);Université de Montréal(蒙特利尔大学)

Ahmed Touati, Pascal Vincent

摘要:

\textit{Smoothed Bellman Error Embedding}算法~ \citep{dai2018sbeed}(SBEED)是一种可证明收敛的具有一般非线性函数逼近的强化学习算法。它已经成功地实现了神经网络,并取得了很好的经验结果。在这项工作中,我们研究了批处理模式强化学习中SBEED的理论行为。我们证明了一个接近最佳的性能保证,依赖于所使用的函数类的表示能力和一个紧的概念的分布偏移。我们的研究结果改善了在~ \citet{dai2018sbeed}中的SBEED在规划范围和样本大小上的依赖性方面的先验保证。我们的分析建立在最近的工作~ \citet{Xie2020}研究相关的算法MSBO,这可以被解释为一个\textit{non-smooth}对应的SBEED。

英文摘要:

The \textit{Smoothed Bellman Error Embedding} algorithm~\citep{dai2018sbeed}, known as SBEED, was proposed as a provably convergent reinforcement learning algorithm with general nonlinear function approximation. It has been successfully implemented with neural networks and achieved strong empirical results. In this work, we study the theoretical behavior of SBEED in batch-mode reinforcement learning. We prove a near-optimal performance guarantee that depends on the representation power of the used function classes and a tight notion of the distribution shift. Our results improve upon prior guarantees for SBEED in ~\citet{dai2018sbeed} in terms of the dependence on the planning horizon and on the sample size. Our analysis builds on the recent work of ~\citet{Xie2020} which studies a related algorithm MSBO, that could be interpreted as a \textit{non-smooth} counterpart of SBEED.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑