离线目标条件强化学习中的潜在表示对齐
Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning
AI总结：
针对离线目标条件强化学习中价值函数错误泛化的瓶颈，提出潜在对齐价值学习（LAVL）算法，通过潜在表示价值泛化与分层规划的统一框架，在OGBench的22个数据集上20个取得最优性能。
摘要：
离线目标条件强化学习（GCRL）提供了一个从固定数据集获取目标达成策略的实用框架。然而，在长视野任务中学习可靠的目标条件价值函数仍然具有挑战性。在本文中，我们指出目标条件价值函数中的错误泛化是一个根本性瓶颈，并证明在价值函数中引入适当的归纳偏置对于解决该瓶颈至关重要。基于这些发现，我们提出了潜在对齐价值学习（LAVL），一种离线GCRL算法，它将基于潜在表示的价值泛化与分层规划集成在一个统一框架中。在OGBench上的大量实验表明，LAVL持续优于现有的离线GCRL方法，在22个数据集中的20个上取得了最高性能。值得注意的是，LAVL在长视野任务和轨迹拼接数据集上表现出强大的性能，而先前的方法在这些任务上性能显著下降。我们的代码可在https://github.com/oh-lab/LAVL.git获取。
英文摘要：
Offline goal-conditioned reinforcement learning (GCRL) provides a practical framework for obtaining goal-reaching policies from fixed datasets. However, learning a reliable goal-conditioned value function in long-horizon tasks remains challenging. In this paper, we identify erroneous generalization in goal-conditioned value functions as a fundamental bottleneck, and demonstrate that appropriate inductive bias in the value function is crucial for addressing the bottleneck. Building on these findings, we propose Latent-Aligned Value Learning (LAVL), an offline GCRL algorithm that integrates latent-representation-based value generalization with hierarchical planning in a unified framework. Extensive experiments on OGBench demonstrate that LAVL consistently outperforms existing offline GCRL methods, achieving the highest performance on 20 out of 22 datasets. Notably, LAVL exhibits strong performance in long-horizon tasks and trajectory stitching datasets, where prior methods suffer significant performance degradation. Our code is available at https://github.com/oh-lab/LAVL.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
