arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.01097cs.CVcs.LGcs.RO
收藏

函数对应问题

The Functional Correspondence Problem

Carnegie Mellon University(卡内基梅隆大学)

Zihang Lai, Senthil Purushwalkam, Abhinav Gupta

摘要:

在视觉数据中找到对应关系的能力是大多数计算机视觉任务的本质。但什么是正确的对应关系呢?视觉对应的任务是很好地定义为两个不同的图像相同的对象实例。在属于同一类别的对象的两个图像的情况下,在大多数情况下,视觉对应是合理定义的。但是,两个完全不同类别的对象之间的对应关系又如何呢?一只鞋和一个瓶子是否存在任何通信?受人类能力的启发:(a)概括语义范畴之外;(b)推断功能启示,本文介绍了功能对应的问题。给定两个物体的图像,我们问一个简单的问题:对于给定的任务,这两个图像之间的对应关系是什么?例如,对于敲打或倾倒的任务,瓶子和鞋子之间的对应关系是什么。我们引入了一个新的数据集:FunKPoint,它具有10个任务和20个对象类别的地面真实对应关系。我们还介绍了一个模块化的任务驱动的表示攻击这个问题,并证明我们学到的表示是有效的这项任务。但最重要的是,因为我们的监督信号不受语义的约束,我们表明,我们的学习表示可以更好地推广Few-Shot分类问题。我们希望这篇论文能激励我们的社区超越语义,更多地关注机器人任务的跨类别泛化和学习表示。

英文摘要:

The ability to find correspondences in visual data is the essence of most computer vision tasks. But what are the right correspondences? The task of visual correspondence is well defined for two different images of same object instance. In case of two images of objects belonging to same category, visual correspondence is reasonably well-defined in most cases. But what about correspondence between two objects of completely different category -- e.g., a shoe and a bottle? Does there exist any correspondence? Inspired by humans' ability to: (a) generalize beyond semantic categories and; (b) infer functional affordances, we introduce the problem of functional correspondences in this paper. Given images of two objects, we ask a simple question: what is the set of correspondences between these two images for a given task? For example, what are the correspondences between a bottle and shoe for the task of pounding or the task of pouring. We introduce a new dataset: FunKPoint that has ground truth correspondences for 10 tasks and 20 object categories. We also introduce a modular task-driven representation for attacking this problem and demonstrate that our learned representation is effective for this task. But most importantly, because our supervision signal is not bound by semantics, we show that our learned representation can generalize better on few-shot classification problem. We hope this paper will inspire our community to think beyond semantics and focus more on cross-category generalization and learning representations for robotics tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑