arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.03586cs.CV
收藏

一种新的基于重建对齐的目标视点估计方法

Novel Object Viewpoint Estimation through Reconstruction Alignment

University of Michigan(密歇根大学)

Mohamed El Banani, Jason J. Corso, David F. Fouhey

摘要:

本文的目的是估计一个新的对象的观点。标准视点估计方法通常在此任务上失败,这是由于它们依赖于用于对准的3D模型或大量类特定的训练数据及其对应的规范姿态。我们通过学习重建和对齐方法来克服这些限制。我们的关键见解是,尽管我们没有明确的3D模型或预定义的规范姿势,但我们仍然可以学习估计对象在观看者框架中的形状,然后使用图像来提供我们的参考模型或规范姿势。特别是,我们提出学习两个网络:第一个将图像映射到3D几何感知特征瓶颈,并通过图像到图像的平移损失进行训练;第二个学习两个特征实例是否对齐。在测试时,我们的模型会找到最能将测试图像的瓶颈特征与参考图像对齐的相对变换。我们通过在不同的数据集上进行推广,分析我们不同模块的影响,并对学习到的特征进行定性分析,以确定正在学习哪些表示来进行对齐,从而评估我们在新对象视点估计上的方法。

英文摘要:

The goal of this paper is to estimate the viewpoint for a novel object. Standard viewpoint estimation approaches generally fail on this task due to their reliance on a 3D model for alignment or large amounts of class-specific training data and their corresponding canonical pose. We overcome those limitations by learning a reconstruct and align approach. Our key insight is that although we do not have an explicit 3D model or a predefined canonical pose, we can still learn to estimate the object's shape in the viewer's frame and then use an image to provide our reference model or canonical pose. In particular, we propose learning two networks: the first maps images to a 3D geometry-aware feature bottleneck and is trained via an image-to-image translation loss; the second learns whether two instances of features are aligned. At test time, our model finds the relative transformation that best aligns the bottleneck features of our test image to a reference image. We evaluate our method on novel object viewpoint estimation by generalizing across different datasets, analyzing the impact of our different modules, and providing a qualitative analysis of the learned features to identify what representations are being learnt for alignment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑