面向自动驾驶的强化学习智能体的风险感知奖励塑形
Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving
AI总结：
针对自动驾驶强化学习奖励函数难确定、传统方法缺风险感知的问题，提出风险感知奖励塑形方法，新增鼓励探索、惩罚风险行为的奖励项，仿真验证对多类智能体有效，且PPO方法配合效果最佳。
摘要：
强化学习（RL）是自动驾驶中运动规划的有效方法，可利用与环境的交互数据自动学习最优驾驶策略。然而，对RL智能体性能至关重要的奖励函数却难以确定。传统工作主要关注对安全驾驶状态给予奖励，但未纳入对车辆风险驾驶行为的感知。本文研究如何利用风险感知奖励塑形来提升自动驾驶中RL智能体的训练与测试性能。基于实践中规定通用自动驾驶安全规范的核心要求，我们提出了额外的塑形奖励项，用于鼓励探索并惩罚风险驾驶行为。在OpenAI Gym中的仿真研究表明，风险感知奖励塑形对各类RL智能体均有优势。此外，我们指出近端策略优化（PPO）很可能是与风险感知奖励塑形配合效果最佳的RL方法。
英文摘要：
Reinforcement learning (RL) is an effective approach to motion planning in autonomous driving, where an optimal driving policy can be automatically learned using the interaction data with the environment. Nevertheless, the reward function for an RL agent, which is significant to its performance, is challenging to be determined. The conventional work mainly focuses on rewarding safe driving states but does not incorporate the awareness of risky driving behaviors of the vehicles. In this paper, we investigate how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. Based on the essential requirements that prescribe the safety specifications for general autonomous driving in practice, we propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors. A simulation study in OpenAI Gym indicates the advantage of risk-aware reward shaping for various RL agents. Also, we point out that proximal policy optimization (PPO) is likely to be the best RL method that works with risk-aware reward shaping.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
