arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

通过量化动作表示的离线强化学习学习可靠泊车策略

Learning Reliable Parking Policies via Offline Reinforcement Learning with Quantized Action Representations

Zewei Yang, Zengqi Peng, Jun Ma

arXiv 2609.19894首次发表:更新:

发表机构

The Hong Kong University of Science and Technology (Guangzhou); The Hong Kong University of Science and Technology(香港科技大学(广州); 香港科技大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

针对城市泊车中杂乱空间与交互不确定性,提出航路点级离线强化学习框架,结合量化动作令牌与保守Q学习,在CARLA中实现最高成功率并可靠泛化。

AI 中文摘要

泊车是自动驾驶车辆在城市环境中运行的一项常规但安全关键的任务。然而,杂乱且结构松散的泊车空间,加上周围车辆带来的交互不确定性,阻碍了可靠机动动作的生成。为解决这些挑战,我们开发了一个用于交互感知的自主泊车的航路点级离线强化学习框架。具体而言,我们通过分层专家回放和旋转航路点增强构建了一个专门的泊车数据集,涵盖了非交互场景和交互场景。策略随后以紧凑的状态表示作为条件,其中基于激光雷达的障碍物特征通过特征级线性调制适应目标位姿。一个状态条件的分词器进一步将连续航路点序列量化为离散动作令牌,在此基础上执行保守Q学习以抑制对支持不足动作的价值过估计。在高保真CARLA模拟器中进行了广泛的闭环实验。所提出的框架在所有基线中取得了最高的泊车成功率,并可靠地迁移到未见过的泊车位。

英文摘要

Parking is a routine yet safety-critical task for autonomous vehicles operating in urban environments. However, cluttered and weakly structured parking spaces, compounded by the interactive uncertainty from surrounding vehicles, hinder reliable maneuver generation. To address these challenges, we develop a waypoint-level offline reinforcement learning framework for interaction-aware autonomous parking. Specifically, a dedicated parking dataset is constructed from hierarchical expert rollouts with rotational waypoint augmentation, covering both non-interactive scenarios and interactive ones. The policy is then conditioned on a compact state representation, in which LiDAR-based obstacle features are adapted to the target pose via feature-wise linear modulation. A state-conditioned tokenizer further quantizes continuous waypoint sequences into discrete action tokens, over which conservative Q-learning is performed to suppress value overestimation on poorly supported actions. Extensive closed-loop experiments are conducted in the high-fidelity CARLA simulator. The proposed framework attains the highest parking success rate among all baselines and transfers reliably to unseen parking slots.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑