arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.09368cs.CV
收藏

Patch2CAD:用于从单个图像中进行自然形状检索的逐块嵌入学习

Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image

Google Research, Brain Team(谷歌研究院,大脑团队);Technical University of Munich(慕尼黑工业大学)

Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, Angela Dai

摘要:

从RGB图像输入的物体形状的3D感知是语义场景理解的基础,在我们的空间三维现实世界环境中基于图像的感知。为了实现对象的图像视图和3D形状之间的映射,我们利用现有的大型数据库中的CAD模型先验,并提出了一种新的方法来构建一个联合嵌入空间之间的2D图像和3D CAD模型在一个补丁明智的方式-建立一个对象的图像视图的补丁和补丁的CAD几何形状之间的对应关系。这使得能够进行零件相似性推理,以检索与新图像视图相似的CAD,而无需在数据库中进行精确匹配。我们的补丁嵌入提供了更强大的CAD检索的形状估计在我们的端到端的CAD模型的形状和姿态的估计在一个单一的输入图像中检测到的对象。在野外,复杂的图像从ScanNet的实验表明,我们的方法是更强大的比最先进的在现实世界中的场景没有任何确切的CAD匹配。

英文摘要:

3D perception of object shapes from RGB image input is fundamental towards semantic scene understanding, grounding image-based perception in our spatially 3-dimensional real-world environments. To achieve a mapping between image views of objects and 3D shapes, we leverage CAD model priors from existing large-scale databases, and propose a novel approach towards constructing a joint embedding space between 2D images and 3D CAD models in a patch-wise fashion -- establishing correspondences between patches of an image view of an object and patches of CAD geometry. This enables part similarity reasoning for retrieving similar CADs to a new image view without exact matches in the database. Our patch embedding provides more robust CAD retrieval for shape estimation in our end-to-end estimation of CAD model shape and pose for detected objects in a single input image. Experiments on in-the-wild, complex imagery from ScanNet show that our approach is more robust than state of the art in real-world scenarios without any exact CAD matches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑