基于分层强化学习的安全无地图导航与拥塞估计
Hierarchical Reinforcement Learning for Safe Mapless Navigation with Congestion Estimation
AI总结：
针对无地图导航易陷入局部极小区域的问题，提出融合拥塞估计子目标机制、安全强化学习低层规划与障碍物编码的HRL框架，并通过仿真和TurtleBot3实验验证其性能与泛化能力。
摘要：
基于强化学习的无地图导航具有巨大潜力。然而，它在存在局部极小区域的室内环境中面临挑战。本文提出一种安全的无地图导航框架，利用分层强化学习（HRL）来增强机器人通过此类区域的导航能力。高层策略创建子目标以引导导航过程。值得注意的是，我们开发了一种考虑环境拥塞的子目标更新机制，可有效避免机器人被困在局部极小区域。通过安全强化学习训练的低层运动规划策略，根据获得的子目标输出实时控制指令。具体而言，为增强机器人的环境感知能力，我们引入了一种新的障碍物编码方法，用于评估障碍物对机器人运动规划的影响。为验证基于HRL的导航框架的性能，我们在办公室、家庭和餐厅环境中进行了仿真。结果表明，基于HRL的导航框架在静态和动态场景中均表现优异。最后，我们在TurtleBot3机器人上实现了基于HRL的导航框架以进行物理验证实验，展现出其强大的泛化能力。
英文摘要：
Reinforcement learning-based mapless navigation holds significant potential. However, it faces challenges in indoor environments with local minima area. This paper introduces a safe mapless navigation framework utilizing hierarchical reinforcement learning (HRL) to enhance navigation through such areas. The high-level policy creates a sub-goal to direct the navigation process. Notably, we have developed a sub-goal update mechanism that considers environment congestion, efficiently avoiding the entrapment of the robot in local minimum areas. The low-level motion planning policy, trained through safe reinforcement learning, outputs real-time control instructions based on acquired sub-goal. Specifically, to enhance the robot's environmental perception, we introduce a new obstacle encoding method that evaluates the impact of obstacles on the robot's motion planning. To validate the performance of our HRL-based navigation framework, we conduct simulations in office, home, and restaurant environments. The findings demonstrate that our HRL-based navigation framework excels in both static and dynamic scenarios. Finally, we implement the HRL-based navigation framework on a TurtleBot3 robot for physical validation experiments, which exhibits its strong generalization capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
