Mollified Value Learning
Mollified Value Learning
AI总结：
针对离线目标条件强化学习中值函数估计困难的问题，提出一种通过空间测度聚合约束（而非逐点微分约束）来诱导距离类值几何的方法，称为Mollified Value Learning（MVL），在导航和高维机器人操作任务中提升了目标达成性能。
摘要：
离线目标条件强化学习（GCRL）从静态数据集中学习达到目标的行为，但在有限的状态-动作覆盖下，准确的值估计仍然具有挑战性。现有的物理信息方法通过施加由Hamilton-Jacobi-Bellman（HJB）最优性原理导出的逐点距离类几何约束（通常通过一阶偏微分方程如Eikonal方程）来解决这一问题。然而，通过显式微分结构强制局部一致性在复杂高维环境中可能变得不稳定。我们的关键洞察是，将距离类约束重新解释为局部空间测度上的期望。通过在该测度上聚合约束而非逐点评估，目标函数充当空间平滑器（mollifier），在无需昂贵微分算子的情况下诱导出距离类值几何。我们称之为Mollified Value Learning（MVL）。在导航和高维机器人操作任务上的实验表明，当与隐式值表示学习方法结合使用时，MVL学习到结构化的值表示，提高了目标达成性能。开源代码可在https://github.com/HrishikeshVish/MVL获取。
英文摘要：
Offline goal-conditioned reinforcement learning (GCRL) learns goal-reaching behaviors from static datasets, but accurate value estimation remains challenging under limited state-action coverage. Existing physics-informed approaches address this by imposing pointwise distance-like geometric constraints derived from Hamilton--Jacobi--Bellman (HJB) optimality principles, often through first-order partial differential equations such as the Eikonal equation. However, enforcing local consistency through explicit differential structure can become unstable in complex, high-dimensional environments. Our key insight is to instead reinterpret distance-like constraints as an expectation over a local spatial measure. By aggregating constraints over this measure rather than evaluating them pointwise, the objective acts as a spatial mollifier, inducing distance-like value geometry without requiring expensive differential operators. We refer to this as Mollified Value Learning (MVL). Experiments across navigation and high-dimensional robotic manipulation tasks show that MVL learns structured, value representations, improving goal-reaching performance, when used with implicit value representation learning methods. Open-source codes are available at https://github.com/HrishikeshVish/MVL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
