通过奖励塑形实现机器人导航深度强化学习的泛化性
Generalization in Deep Reinforcement Learning for Robotic Navigation by Reward Shaping
AI总结：
针对DRL机器人局部导航易陷局部极小值、泛化性不足的问题，提出融入训练阶段地图信息的新型奖励函数，结合SAC算法训练，经仿真和实机实验验证在避障和规避局部极小值上性能更优。
摘要：
本文研究了DRL（深度强化学习）算法在局部导航问题中的应用：机器人仅配备LiDAR（激光雷达）等有限距离外感知传感器，在未知且杂乱的工作空间中向目标位置移动。基于DRL的避障策略虽有优势，但由于其学习合适动作的能力受限于传感器范围，极易陷入局部极小值。由于大多数机器人在非结构化环境中执行任务，因此研究能够避免局部极小值、尤其在未训练场景下具备泛化能力的局部导航策略具有重要意义。为此，我们提出一种新型奖励函数，该函数融入了训练阶段获取的地图信息，提升了智能体斟酌最优行动方案的能力。同时，我们采用SAC（软演员评论家）算法训练我们的ANN（人工神经网络），该算法比现有前沿文献中的其他算法更有效。一系列sim-to-sim（仿真到仿真）和sim-to-real（仿真到现实）实验表明，我们提出的奖励函数结合SAC算法在局部极小值规避和避障方面优于对比方法。
英文摘要：
In this paper, we study the application of DRL algorithms in the context of local navigation problems, in which a robot moves towards a goal location in unknown and cluttered workspaces equipped only with limited-range exteroceptive sensors, such as LiDAR. Collision avoidance policies based on DRL present some advantages, but they are quite susceptible to local minima, once their capacity to learn suitable actions is limited to the sensor range. Since most robots perform tasks in unstructured environments, it is of great interest to seek generalized local navigation policies capable of avoiding local minima, especially in untrained scenarios. To do so, we propose a novel reward function that incorporates map information gained in the training stage, increasing the agent's capacity to deliberate about the best course of action. Also, we use the SAC algorithm for training our ANN, which shows to be more effective than others in the state-of-the-art literature. A set of sim-to-sim and sim-to-real experiments illustrate that our proposed reward combined with the SAC outperforms the compared methods in terms of local minima and collision avoidance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
