arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.14641cs.LGcs.AIcs.RO
收藏

学习乐观地计划:通过潜在模型集成进行不确定性引导的深度探索

Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);MIT LIDS(麻省理工学院信息与决策系统实验室)

Tim Seyde, Wilko Schwarting, Sertac Karaman, Daniela Rus

摘要:

通过交互学习复杂的机器人行为需要结构化的探索。规划应针对具有优化长期绩效潜力的互动,同时只在有利于这一目标的情况下减少不确定性。本文介绍了潜在乐观价值探索(LOVE),这是一种在面对不确定的长期回报时通过乐观进行深度探索的策略。我们将潜在世界模型与价值函数估计相结合,通过集成来预测无限期收益并恢复相关的不确定性。然后,该策略在置信上限(UCB)目标上进行训练,以识别和选择最有希望提高长期性能的交互。我们将LOVE应用于连续动作空间中的视觉机器人控制任务,并与最先进的和其他探索目标相比,平均提高了20%以上的样本效率。在稀疏且难以探索的环境中,我们的平均改善率超过30%。

英文摘要:

Learning complex robot behaviors through interaction requires structured exploration. Planning should target interactions with the potential to optimize long-term performance, while only reducing uncertainty where conducive to this objective. This paper presents Latent Optimistic Value Exploration (LOVE), a strategy that enables deep exploration through optimism in the face of uncertain long-term rewards. We combine latent world models with value function estimation to predict infinite-horizon returns and recover associated uncertainty via ensembling. The policy is then trained on an upper confidence bound (UCB) objective to identify and select the interactions most promising to improve long-term performance. We apply LOVE to visual robot control tasks in continuous action spaces and demonstrate on average more than 20% improved sample efficiency in comparison to state-of-the-art and other exploration objectives. In sparse and hard to explore environments we achieve an average improvement of over 30%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑