arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1604.01360cs.CVcs.AIcs.RO
收藏

好奇的机器人:通过物理交互学习视觉表征

The Curious Robot: Learning Visual Representations via Physical Interactions

Carnegie Mellon University(卡内基梅隆大学)

Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta

摘要:

什么是正确的监督信号,以训练视觉表示?计算机视觉中的当前方法使用来自ImageNet等数据集的类别标签来训练ConvNets。然而,在生物制剂的情况下,视觉表示学习不需要数百万个语义标签。我们认为,生物制剂使用物理与世界的相互作用,以学习视觉表示不像目前的视觉系统,只是使用被动的观察(从网络上下载的图像和视频)。例如,婴儿推物体,戳它们,把它们放进嘴里,把它们扔出去,以学习表征。为了实现这一目标,我们在Baxter平台上构建了首批系统之一,该系统可以在桌面环境中推、戳、抓和观察物体。它使用四种不同类型的物理交互来收集超过13万个数据点,每个数据点都为共享的ConvNet架构提供监督,使我们能够学习可视化表示。我们通过观察神经元激活和对这个学习的表示进行最近邻检索来显示学习的表示的质量。定量地,我们评估了我们学习的ConvNet对图像分类任务的影响,并显示了与没有外部数据的学习相比的改进。最后,在实例检索任务上,我们的网络在recall@1上的表现优于ImageNet网络3%。

英文摘要:

What is the right supervisory signal to train visual representations? Current approaches in computer vision use category labels from datasets such as ImageNet to train ConvNets. However, in case of biological agents, visual representation learning does not require millions of semantic labels. We argue that biological agents use physical interactions with the world to learn visual representations unlike current vision systems which just use passive observations (images and videos downloaded from web). For example, babies push objects, poke them, put them in their mouth and throw them to learn representations. Towards this goal, we build one of the first systems on a Baxter platform that pushes, pokes, grasps and observes objects in a tabletop environment. It uses four different types of physical interactions to collect more than 130K datapoints, with each datapoint providing supervision to a shared ConvNet architecture allowing us to learn visual representations. We show the quality of learned representations by observing neuron activations and performing nearest neighbor retrieval on this learned representation. Quantitatively, we evaluate our learned ConvNet on image classification tasks and show improvements compared to learning without external data. Finally, on the task of instance retrieval, our network outperforms the ImageNet network on recall@1 by 3%

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑