arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.08925cs.CVcs.LG
收藏

基于图像到二十面体投影的单视图$\mathrm{SO}(3)$对象推理

Image to Icosahedral Projection for $\mathrm{SO}(3)$ Object Reasoning from Single-View Images

Northeastern University(东北大学)

David Klee, Ondrej Biza, Robert Platt, Robin Walters

摘要:

基于2D图像的3D对象的推理是具有挑战性的,这是由于从不同方向观看对象所引起的外观变化。诸如对象分类之类的任务对于3D旋转是不变的,而诸如姿态估计之类的其他任务是等变的。然而,施加等方差作为模型约束对于2D图像输入通常是不可能的,因为我们没有关于图像在平面外对象旋转下如何变化的先验模型。目前存在的唯一$\mathrm{SO}(3)$ -等变模型需要点云或体素输入,而不是2D图像。在本文中,我们提出了一种新的架构的基础上二十面体群卷积的原因,在$\mathrm{SO(3)}$通过学习的投影输入图像到一个二十面体。得到的模型与$\mathrm{SO}(3)$中的旋转近似等变。我们将此模型应用于物体姿态估计和形状分类任务,并发现它优于合理的基线。项目网址:\url{https://dmklee.github.io/image2icosahedral}

英文摘要:

Reasoning about 3D objects based on 2D images is challenging due to variations in appearance caused by viewing the object from different orientations. Tasks such as object classification are invariant to 3D rotations and other such as pose estimation are equivariant. However, imposing equivariance as a model constraint is typically not possible with 2D image input because we do not have an a priori model of how the image changes under out-of-plane object rotations. The only $\mathrm{SO}(3)$-equivariant models that currently exist require point cloud or voxel input rather than 2D images. In this paper, we propose a novel architecture based on icosahedral group convolutions that reasons in $\mathrm{SO(3)}$ by learning a projection of the input image onto an icosahedron. The resulting model is approximately equivariant to rotation in $\mathrm{SO}(3)$. We apply this model to object pose estimation and shape classification tasks and find that it outperforms reasonable baselines. Project website: \url{https://dmklee.github.io/image2icosahedral}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑