arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.08511cs.RO

视觉驱动的无人机河流跟随:基于安全强化学习的基准测试

Vision-driven UAV River Following: Benchmarking with Safe Reinforcement Learning

School of Mechanical Engineering, Purdue University(机械工程学院,普渡大学)

Zihan Wang, Nina Mahmoudian

AI总结:

本研究在Unity模拟环境中对无人机视觉驱动河流跟随任务的安全强化学习算法进行基准测试,发现语义增强图像编码和on-policy算法表现最佳,且方法可扩展至自主水面车辆。

摘要:

在本研究中,我们在基于Unity的逼真模拟环境中,对用于无人机(UAV)视觉驱动河流跟随任务的安全强化学习(Safe RL)算法进行了全面基准测试。我们实证验证了语义增强图像编码方法的有效性,基于相对熵和水像素重建质量评估了其优越性。在重建损失的指导下确定编码维度,有助于构建更紧凑的状态表示,从而促进Safe RL策略的训练。在所有基准测试的Safe RL算法中,我们发现策略空间中的一阶约束优化在奖励获取和安全合规之间实现了最佳平衡。值得注意的是,我们的结果显示,on-policy算法在训练和测试环境中均始终优于off-policy和基于模型的同类算法。重要的是,基准测试结果和视觉编码方法不仅限于无人机,还适用于在受限水域进行自主导航的自主水面车辆(ASV)。

英文摘要:

In this study, we conduct a comprehensive benchmark of the Safe Reinforcement Learning (Safe RL) algorithms for the task of vision-driven river following of Unmanned Aerial Vehicle (UAV) in a Unity-based photo-realistic simulation environment. We empirically validate the effectiveness of semantic-augmented image encoding method, assessing its superiority based on Relative Entropy and the quality of water pixel reconstruction. The determination of the encoding dimension, guided by reconstruction loss, contributes to a more compact state representation, facilitating the training of Safe RL policies. Across all benchmarked Safe RL algorithms, we find that First Order Constrained Optimization in Policy Space achieves the optimal balance between reward acquisition and safety compliance. Notably, our results reveal that on-policy algorithms consistently outperform both off-policy and model-based counterparts in both training and testing environments. Importantly, the benchmarking outcomes and the vision encoding methodology extend beyond UAVs, and are applicable to Autonomous Surface Vehicles (ASVs) engaged in autonomous navigation in confined waters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑