离线目标条件强化学习与投影拟度规划
Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planning
AI总结：
本文提出ProQ框架，通过学习非对称距离和结合拉格朗日检测器，实现目标条件强化学习中子目标生成与长期目标达成的稳健控制。
摘要：
离线目标条件强化学习旨在从之前收集的轨迹中训练智能体达到指定目标。扩展到长期任务仍具挑战性，特别是由于价值估计误差的累积。原理性的几何方法提供了潜在的解决方案。受此启发，我们引入了投影拟度规划（ProQ），一种组合框架，它学习一个非对称距离，然后将其重新利用，首先作为排斥能量迫使稀疏的关键点集在学习的潜在空间中均匀分布，其次作为结构化方向成本引导向近邻子目标。特别是，ProQ将此几何与拉格朗日分布外检测器结合，以确保学习的关键点保持在可到达区域。通过统一度量学习、关键点覆盖和目标条件控制，我们的方法产生了有意义的子目标，并在多样化的导航基准上稳健地驱动长期目标达成。
英文摘要：
Offline Goal-Conditioned Reinforcement Learning seeks to train agents to reach specified goals from previously collected trajectories. Scaling that promises to long-horizon tasks remains challenging, notably due to compounding value-estimation errors. Principled geometric offers a potential solution to address these issues. Following this insight, we introduce Projective Quasimetric Planning (ProQ), a compositional framework that learns an asymmetric distance and then repurposes it, firstly as a repulsive energy forcing a sparse set of keypoints to uniformly spread over the learned latent space, and secondly as a structured directional cost guiding towards proximal sub-goals. In particular, ProQ couples this geometry with a Lagrangian out-of-distribution detector to ensure the learned keypoints stay within reachable areas. By unifying metric learning, keypoint coverage, and goal-conditioned control, our approach produces meaningful sub-goals and robustly drives long-horizon goal-reaching on diverse a navigation benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
