arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.04036cs.RO
收藏

深度强化学习用于无先验地图的自主地面车辆探索

Deep Reinforcement Learning for Autonomous Ground Vehicle Exploration Without A-Priori Maps

Virginia Tech(弗吉尼亚理工大学)

Shathushan Sivashangaran, Azim Eskandarian

AI总结:

本研究提出基于Actor-Critic的深度强化学习框架,使自主地面车辆无需先验地图即可从原始传感器数据学习策略,实现快速无碰撞探索,并验证了其适应未知环境的可扩展性。

摘要:

自主地面车辆(AGV)因其能够在危险环境中以最少的人工操作员输入进行作业,成为广泛应用的重要工具。有效的运动规划对于AGV的成功运行至关重要。传统的运动规划算法依赖于对环境特征的先验知识,在信息匮乏、动态变化的环境中(如发生火灾和地震等紧急危险的区域,以及火星上的隧道和熔岩管等未探索的地下环境)效用有限。我们提出了一种深度强化学习(DRL)框架,用于无先验地图的智能AGV探索,利用Actor-Critic DRL算法直接从原始传感器数据中学习连续和高维动作空间中的策略。该DRL架构包括用于评论家和演员表示的前馈神经网络,其中演员网络根据当前状态输入制定线性和角速度控制动作,评论家网络学习并估计Q值以最大化累积奖励。我们训练并比较了三种离策略DRL算法,即DDPG、TD3和SAC,在两种复杂度不同的环境中进行,并在第三种环境中进一步评估,该环境没有先前的训练或地图特征知识。结果表明,智能体在每个训练期结束时学会了最优策略,能够规划快速、无碰撞的探索轨迹,并且该框架是可扩展的,能够在无需更改网络架构或超参数的情况下适应未知环境。最佳算法还在一个逼真的3D环境中进行了进一步评估。

英文摘要:

Autonomous Ground Vehicles (AGVs) are essential tools for a wide range of applications stemming from their ability to operate in hazardous environments with minimal human operator input. Effective motion planning is paramount for successful operation of AGVs. Conventional motion planning algorithms are dependent on prior knowledge of environment characteristics and offer limited utility in information poor, dynamically altering environments such as areas where emergency hazards like fire and earthquake occur, and unexplored subterranean environments such as tunnels and lava tubes on Mars. We propose a Deep Reinforcement Learning (DRL) framework for intelligent AGV exploration without a-priori maps utilizing Actor-Critic DRL algorithms to learn policies in continuous and high-dimensional action spaces directly from raw sensor data. The DRL architecture comprises feedforward neural networks for the critic and actor representations in which the actor network strategizes linear and angular velocity control actions given current state inputs, that are evaluated by the critic network which learns and estimates Q-values to maximize an accumulated reward. Three off-policy DRL algorithms, DDPG, TD3 and SAC, are trained and compared in two environments of varying complexity, and further evaluated in a third with no prior training or knowledge of map characteristics. The agent is shown to learn optimal policies at the end of each training period to chart quick, collision-free exploration trajectories, and is extensible, capable of adapting to an unknown environment without changes to network architecture or hyperparameters. The best algorithm is further evaluated in a realistic 3D environment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑