arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.08050eess.SYcs.SY
收藏

统一哈密顿-雅可比可达性与强化学习

Unifying Hamilton-Jacobi Reachability and Reinforcement Learning

Delft University of Technology(代尔夫特理工大学)

Prashant Solanki, Isabelle El-Hajj, Jasper van Beers, Erik-Jan van Kampen, Coen de Visser

AI总结:

本文通过提出的运行成本公式统一哈密顿-雅可比可达性与强化学习,证明了旅行成本价值函数是时间依赖哈密顿-雅可比贝尔曼PDE的唯一有界粘性解,并通过实验验证了学习价值函数向半拉格朗日HJB解的收敛性。

摘要:

我们通过提出的运行成本公式统一哈密顿-雅可比(HJ)可达性与强化学习(RL)。我们证明了所得到的旅行成本价值函数是时间依赖的哈密顿-雅可比贝尔曼(HJB)偏微分方程(PDE)的唯一有界粘性解,其负子集等于严格后向可达管。通过正向重参数化和收缩诱导的贝尔曼更新,我们显示小步RL价值迭代的固定点收敛于正向折扣HJB的粘性解。在经典基准上的实验验证了这种联系,通过展示学习价值函数向半拉格朗日HJB解的收敛性,并通过量化状态空间中的近似误差来量化。这些结果支持了理论分析,表明所提出的框架在保持可达性安全语义的同时,仍与深度RL实现兼容。

英文摘要:

We unify Hamilton-Jacobi (HJ) reachability and Reinforcement Learning (RL) through a proposed running cost formulation. We prove that the resultant travel-cost value function is the unique bounded viscosity solution of a time-dependent Hamilton-Jacobi Bellman (HJB) Partial Differential Equation (PDE) with zero terminal data, whose negative sublevel set equals the strict backward-reachable tube. Using a forward reparameterization and a contraction inducing Bellman update, we show that fixed points of small-step RL value iteration converge to the viscosity solution of the forward discounted HJB. Experiments on a classical benchmark validate this connection by demonstrating convergence of learned value functions toward semi-Lagrangian HJB solutions and by quantifying approximation error across the state space. These results empirically support the theoretical analysis, showing that the proposed framework preserves reachability-based safety semantics while remaining compatible with deep RL implementations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑