arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.10934cs.ROcs.AI
收藏

LEAF:前沿的潜在探索

LEAF: Latent Exploration Along the Frontier

University of Toronto(多伦多大学)

Homanga Bharadhwaj, Animesh Garg, Florian Shkurti

摘要:

自我监督的目标建议和达成是探索和有效的策略学习算法的关键组成部分。这种不访问任何Oracle目标抽样分布的自我监督方法需要深入的探索和承诺,以便可以有效地发现长期计划。在本文中,我们提出了一个探索框架,它学习可达状态的动态感知流形。对于一个目标,我们提出的方法确定性地访问一个国家在当前的边界可达到的国家(承诺/达到),然后随机探索,以达到目标(探索)。这将在可达区域的边界附近而不是其内部分配勘探预算。我们的目标是从指定为图像的初始和目标状态的策略学习的挑战性问题,并不假设任何访问的机器人和环境的底层地面实况状态。为了跟踪可到达的潜在状态,我们提出了一个以距离为条件的可达性网络,该网络经过训练以推断一个状态是否可以在指定的潜在空间距离内从另一个状态到达。给定一个初始状态,我们从该状态得到一个可达状态的边界。通过纳入一个课程的采样更容易的目标(更接近开始状态)之前,更困难的目标,我们证明了所提出的自我监督的探索算法,优越的性能相比,现有的基线上一组具有挑战性的机器人environments.https://sites.google.com/view/leaf-exploration

英文摘要:

Self-supervised goal proposal and reaching is a key component for exploration and efficient policy learning algorithms. Such a self-supervised approach without access to any oracle goal sampling distribution requires deep exploration and commitment so that long horizon plans can be efficiently discovered. In this paper, we propose an exploration framework, which learns a dynamics-aware manifold of reachable states. For a goal, our proposed method deterministically visits a state at the current frontier of reachable states (commitment/reaching) and then stochastically explores to reach the goal (exploration). This allocates exploration budget near the frontier of the reachable region instead of its interior. We target the challenging problem of policy learning from initial and goal states specified as images, and do not assume any access to the underlying ground-truth states of the robot and the environment. To keep track of reachable latent states, we propose a distance-conditioned reachability network that is trained to infer whether one state is reachable from another within the specified latent space distance. Given an initial state, we obtain a frontier of reachable states from that state. By incorporating a curriculum for sampling easier goals (closer to the start state) before more difficult goals, we demonstrate that the proposed self-supervised exploration algorithm, superior performance compared to existing baselines on a set of challenging robotic environments.https://sites.google.com/view/leaf-exploration

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑