用于引导机器人探索的视觉可供性预测
Visual Affordance Prediction for Guiding Robot Exploration
AI总结：
本文提出用Transformer在VQ-VAE潜在空间中学习视觉可供性分布，以预测场景中可实现的未来状态，并将其作为目标采样分布引导机器人操作中的视觉目标条件策略探索，展现出对训练分布外物体的组合泛化能力。
摘要：
受人类对可能交互空间的直觉理解以及将这种理解轻松泛化到未见场景的能力启发，我们开发了一种学习视觉可供性以引导机器人探索的方法。给定场景的输入图像，我们推断可通过与其交互实现的合理未来状态分布。我们使用基于Transformer的模型在VQ-VAE的潜在嵌入空间中学习条件分布，并表明这些模型可以使用大规模且多样化的被动数据进行训练，且学习到的模型对训练分布之外的多样物体表现出组合泛化能力。我们展示了训练后的可供性模型如何在机器人操作中的视觉目标条件策略学习期间，通过充当目标采样分布来引导探索。
英文摘要：
Motivated by the intuitive understanding humans have about the space of possible interactions, and the ease with which they can generalize this understanding to previously unseen scenes, we develop an approach for learning visual affordances for guiding robot exploration. Given an input image of a scene, we infer a distribution over plausible future states that can be achieved via interactions with it. We use a Transformer-based model to learn a conditional distribution in the latent embedding space of a VQ-VAE and show that these models can be trained using large-scale and diverse passive data, and that the learned models exhibit compositional generalization to diverse objects beyond the training distribution. We show how the trained affordance model can be used for guiding exploration by acting as a goal-sampling distribution, during visual goal-conditioned policy learning in robotic manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
