使用 QPHIL 进行导航：用于分层隐式 Q-Learning 的量化规划器
Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learning
AI总结：
提出一种基于分层 transformer 和空间量化器的离线强化学习方法 QPHIL，通过显式轨迹拼接解决复杂长距离导航任务中的价值估计误差问题，取得 SOTA 结果。
摘要：
离线强化学习（RL）已成为在各个领域（特别是在复杂导航任务中）进行行为建模时模仿学习的强大替代方案。离线 RL 面临的一个现有挑战是信噪比问题，即如何缓解由于价值估计误差导致的错误策略更新。为此，多项研究已证明分层离线 RL 方法的优势，该方法将高层路径规划与低层路径跟踪解耦。在本研究中，我们提出了一种新颖的基于分层 transformer 的方法，利用学习到的空间量化器。这种量化使得训练更简单的基于区域条件的低层策略成为可能，并简化了规划，将其简化为离散自回归预测。在其他益处中，规划中的区域级推理能够实现显式轨迹拼接，而不是基于含噪价值函数估计的隐式拼接。通过将这种基于 transformer 的规划器与离线 RL 的最新进展相结合，我们提出的方法在复杂的长距离导航环境中取得了最先进的结果。
英文摘要：
Offline Reinforcement Learning (RL) has emerged as a powerful alternative to imitation learning for behavior modeling in various domains, particularly in complex navigation tasks. An existing challenge with Offline RL is the signal-to-noise ratio, i.e. how to mitigate incorrect policy updates due to errors in value estimates. Towards this, multiple works have demonstrated the advantage of hierarchical offline RL methods, which decouples high-level path planning from low-level path following. In this work, we present a novel hierarchical transformer-based approach leveraging a learned quantizer of the space. This quantization enables the training of a simpler zone-conditioned low-level policy and simplifies planning, which is reduced to discrete autoregressive prediction. Among other benefits, zone-level reasoning in planning enables explicit trajectory stitching rather than implicit stitching based on noisy value function estimates. By combining this transformer-based planner with recent advancements in offline RL, our proposed approach achieves state-of-the-art results in complex long-distance navigation environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
