不依赖标记驾驶数据的分层强化学习自动驾驶决策
Hierarchical Reinforcement Learning for Self-Driving Decision-Making without Reliance on Labeled Driving Data
摘要：
自动驾驶汽车的决策通常通过手动编码驾驶员行为的规则或使用监督学习技术模仿驾驶员的操作来解决。两者都依赖于海量的驾驶数据来覆盖所有可能的驾驶场景。提出了一种用于自动驾驶汽车决策的分层强化学习方法，该方法不依赖于大量的标记驾驶数据。该方法综合考虑了纵向和横向的高层机动选择和底层运动控制。首先将驾驶任务分解为车道内行驶、右变道和左变道三种操作，并学习每种操作的子策略。然后，学习主策略以选择要在当前状态下执行的机动策略。所有策略，包括主策略和机动策略，都由完全连接的神经网络表示，并使用异步并行强化学习器（APRL）进行训练，建立从感官输出到驾驶决策的映射。针对每种机动设计了不同的状态空间和奖励函数。我们将该方法应用于高速公路驾驶场景，证明它可以实现自动驾驶汽车的平稳、安全决策。
英文摘要：
Decision making for self-driving cars is usually tackled by manually encoding rules from drivers' behaviors or imitating drivers' manipulation using supervised learning techniques. Both of them rely on mass driving data to cover all possible driving scenarios. This paper presents a hierarchical reinforcement learning method for decision making of self-driving cars, which does not depend on a large amount of labeled driving data. This method comprehensively considers both high-level maneuver selection and low-level motion control in both lateral and longitudinal directions. We firstly decompose the driving tasks into three maneuvers, including driving in lane, right lane change and left lane change, and learn the sub-policy for each maneuver. Then, a master policy is learned to choose the maneuver policy to be executed in the current state. All policies including master policy and maneuver policies are represented by fully-connected neural networks and trained by using asynchronous parallel reinforcement learners (APRL), which builds a mapping from the sensory outputs to driving decisions. Different state spaces and reward functions are designed for each maneuver. We apply this method to a highway driving scenario, which demonstrates that it can realize smooth and safe decision making for self-driving cars.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
