arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.21179math.OC
收藏

从正则化强化学习到连续时间随机控制的离散化误差

Discretization error from regularized Reinforcement Learning to continuous-time stochastic control

Ecole Polytechnique(巴黎综合理工学院);Auburn University(奥本大学);University of California, Los Angeles(加州大学洛杉矶分校)

Huyên Pham, Yuming Paul Zhang, Yuhua Zhu

AI总结:

本文研究正则化离散时间强化学习与连续时间随机最优控制之间的离散化误差,分析其收敛速率以支撑探索性RL策略在连续时间环境中的稳定性与实现。

摘要:

本文建立了正则化离散时间强化学习与连续时间随机最优控制之间的严格联系。具体而言,经典RL算法通常解决正则化离散时间Bellman方程。我们研究离散化误差,即正则化离散时间Bellman方程诱导的最佳策略与底层连续时间随机控制问题的真实最优反馈控制之间的差距。通过推导此差距的定量收敛速率,我们为理解探索性RL策略在连续时间环境中的稳定性和实现提供了严格的基础。

英文摘要:

This paper establishes a rigorous connection between regularized discrete-time reinforcement learning (RL) and continuous-time stochastic optimal control. Specifically, classical RL algorithms are typically solving a regularized discrete-time Bellman equation. We study the discretization error, namely, the gap between the optimal policy induced by the regularized discrete-time Bellman equation and the true optimal feedback control of the underlying continuous-time stochastic control problem. By deriving quantitative convergence rates for this gap, we provide a rigorous foundation for understanding the stability and implementation of exploratory RL policies in stochastic continuous-time environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑