挑战性环境中机器人探索的在线自适应离线策略评估
Off-Policy Evaluation with Online Adaptation for Robot Exploration in Challenging Environments
AI总结：
本文提出OPERE方法，将机器人探索建模为离线策略评估问题，结合离线蒙特卡洛训练、时序差分在线自适应和基于传感器信息覆盖的内在奖励，首次在真实地下与城市环境中实现状态价值预测并提升探索效率。
摘要：
自主探索具有许多重要应用。然而，经典的基于信息增益或基于前沿的探索仅依赖机器人当前状态来确定即时探索目标，缺乏预测未来状态价值的能力，从而导致探索决策效率低下。本文提出一种方法，通过状态价值函数学习状态有多“好”，为机器人在真实世界挑战性环境中的探索提供指导。我们将该工作形式化为机器人探索的离线策略评估（OPE）问题（OPERE）。该方法包括在真实世界数据上进行离线蒙特卡洛训练，并进行时序差分（TD）在线自适应以优化训练得到的价值估计器。我们还设计了一种基于传感器信息覆盖的内在奖励函数，使机器人能够在稀疏外在奖励下获得更多信息。结果表明，我们的方法使机器人能够预测未来状态的价值，从而更好地指导机器人探索。与最先进方法相比，所提算法在预测和探索性能上均取得了更好的表现。据我们所知，本工作首次在具有挑战性的地下和城市环境中，基于真实世界数据集展示了用于机器人探索的价值函数预测。更多细节和演示视频可参见 https://jeffreyyh.github.io/opere/。
英文摘要：
Autonomous exploration has many important applications. However, classic information gain-based or frontier-based exploration only relies on the robot current state to determine the immediate exploration goal, which lacks the capability of predicting the value of future states and thus leads to inefficient exploration decisions. This paper presents a method to learn how "good" states are, measured by the state value function, to provide a guidance for robot exploration in real-world challenging environments. We formulate our work as an off-policy evaluation (OPE) problem for robot exploration (OPERE). It consists of offline Monte-Carlo training on real-world data and performs Temporal Difference (TD) online adaptation to optimize the trained value estimator. We also design an intrinsic reward function based on sensor information coverage to enable the robot to gain more information with sparse extrinsic rewards. Results show that our method enables the robot to predict the value of future states so as to better guide robot exploration. The proposed algorithm achieves better prediction and exploration performance compared with the state-of-the-arts. To the best of our knowledge, this work for the first time demonstrates value function prediction on real-world dataset for robot exploration in challenging subterranean and urban environments. More details and demo videos can be found at https://jeffreyyh.github.io/opere/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
