arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.07246cs.RO
收藏

人在回路深度强化学习及其在自动驾驶中的应用

Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving

Nanyang Technological University(南洋理工大学)

Jingda Wu, Zhiyu Huang, Chao Huang, Zhongxu Hu, Peng Hang, Yang Xing, Chen Lv

摘要:

由于机器智能的智能和能力有限,目前自动驾驶汽车仍然无法处理各种情况并完全取代驾驶员。由于人类在复杂的驾驶场景中表现出很强的鲁棒性和适应性,因此将人类引入人工智能的训练循环中,利用人类智能进一步推进机器学习算法具有重要意义。在这项研究中,一个实时的人类指导为基础的深度强化学习(Hug-DRL)方法开发的自动驾驶的政策培训。利用新设计的人与自动化之间的控制转移机制,在模型训练过程中,人能够在必要时实时的干预并纠正智能体的不合理行为。基于这种人在回路的指导机制,一个改进的行动者-评论家架构与修改的政策和价值网络的发展。Hug-DRL的快速收敛允许将实时人类指导动作融合到智能体的训练循环中,进一步提高深度强化学习的效率和性能。该方法通过40名受试者的人在回路实验进行了验证,并与其他最先进的学习方法进行了比较。结果表明,该方法可以有效地提高人工指导下深度强化学习算法的训练效率和性能,而不需要对参与者的专业知识和经验提出具体要求。

英文摘要:

Due to the limited smartness and abilities of machine intelligence, currently autonomous vehicles are still unable to handle all kinds of situations and completely replace drivers. Because humans exhibit strong robustness and adaptability in complex driving scenarios, it is of great importance to introduce humans into the training loop of artificial intelligence, leveraging human intelligence to further advance machine learning algorithms. In this study, a real-time human-guidance-based deep reinforcement learning (Hug-DRL) method is developed for policy training of autonomous driving. Leveraging a newly designed control transfer mechanism between human and automation, human is able to intervene and correct the agent's unreasonable actions in real time when necessary during the model training process. Based on this human-in-the-loop guidance mechanism, an improved actor-critic architecture with modified policy and value networks is developed. The fast convergence of the proposed Hug-DRL allows real-time human guidance actions to be fused into the agent's training loop, further improving the efficiency and performance of deep reinforcement learning. The developed method is validated by human-in-the-loop experiments with 40 subjects and compared with other state-of-the-art learning approaches. The results suggest that the proposed method can effectively enhance the training efficiency and performance of the deep reinforcement learning algorithm under human guidance, without imposing specific requirements on participant expertise and experience.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑