利用分层潜在技能提升自动驾驶的离线强化学习
Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills
AI总结：
本文提出一种基于分层潜在技能的框架，通过变分自编码器学习驾驶技能并作为动作，增强离线强化学习在自动驾驶中的长时域规划能力，在CARLA上优于强基线。
摘要：
随着各种驾驶模拟器和大规模驾驶数据集的出现，基于学习的车辆规划正受到越来越多的关注。虽然离线强化学习（RL）非常适合这些安全关键任务，但它仍然难以在长时间范围内进行规划。在这项工作中，我们提出了一个基于技能的框架，通过增强离线RL来克服长时域车辆规划的挑战。具体来说，我们设计了一个变分自编码器（VAE），从离线演示中学习技能。为了缓解常见VAE的后验崩溃问题，我们引入了一个双分支序列编码器，以同时捕捉复杂驾驶技能的离散选项和连续变化。最终策略将学习到的技能视为动作，并可以通过任何现成的离线RL算法进行训练。这促进了从逐步骤动作到时间上扩展的技能的关注转移，从而实现了对未来的长期推理。在CARLA上的大量结果表明，我们的模型在训练场景和新场景中均持续优于强基线。额外的可视化和实验证明了提取技能的可解释性和可迁移性。
英文摘要：
Learning-based vehicle planning is receiving increasing attention with the emergence of diverse driving simulators and large-scale driving datasets. While offline reinforcement learning (RL) is well suited for these safety-critical tasks, it still struggles to plan over extended periods. In this work, we present a skill-based framework that enhances offline RL to overcome the long-horizon vehicle planning challenge. Specifically, we design a variational autoencoder (VAE) to learn skills from offline demonstrations. To mitigate posterior collapse of common VAEs, we introduce a two-branch sequence encoder to capture both discrete options and continuous variations of the complex driving skills. The final policy treats learned skills as actions and can be trained by any off-the-shelf offline RL algorithms. This facilitates a shift in focus from per-step actions to temporally extended skills, thereby enabling long-term reasoning into the future. Extensive results on CARLA prove that our model consistently outperforms strong baselines at both training and new scenarios. Additional visualizations and experiments demonstrate the interpretability and transferability of extracted skills.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
