arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.04153cs.CVcs.LG
收藏

基于原始图网络的单幅图像三维目标估计

Single Image 3D Object Estimation with Primitive Graph Networks

ShanghaiTech University(上海科技大学);Chinese Academy of Sciences(中国科学院);Baidu Inc.(百度公司);KU Leuven(鲁汶大学)

Qian He, Desen Zhou, Bo Wan, Xuming He

摘要:

从单个图像(RGB或深度)重建3D对象是视觉场景理解中的基本问题,但由于其在现实世界场景中的不适定性质和复杂性而仍然具有挑战性。为了解决这些挑战,我们采用了一个基于连续的表示三维对象,并提出了一个两阶段的图网络连续的三维对象估计,其中包括一个顺序的建议模块和图推理模块。给定一个2D图像,我们的建议模块首先从输入图像中生成一个具有局部特征关注的3D基元序列。然后,图形推理模块对基元图执行联合推理,以捕获每个基元的全局形状上下文。这种框架能够在3D结构恢复期间考虑丰富的几何和语义约束,即使在具有挑战性的观看条件下也能产生具有更连贯结构的3D对象。我们以阶段策略训练整个图神经网络,并在三个基准上对其进行评估:Pix3D,ModelNet和NYU Depth V2。大量的实验表明,我们的方法优于以前的国家的艺术有相当大的利润。

英文摘要:

Reconstructing 3D object from a single image (RGB or depth) is a fundamental problem in visual scene understanding and yet remains challenging due to its ill-posed nature and complexity in real-world scenes. To address those challenges, we adopt a primitive-based representation for 3D object, and propose a two-stage graph network for primitive-based 3D object estimation, which consists of a sequential proposal module and a graph reasoning module. Given a 2D image, our proposal module first generates a sequence of 3D primitives from input image with local feature attention. Then the graph reasoning module performs joint reasoning on a primitive graph to capture the global shape context for each primitive. Such a framework is capable of taking into account rich geometry and semantic constraints during 3D structure recovery, producing 3D objects with more coherent structure even under challenging viewing conditions. We train the entire graph neural network in a stage-wise strategy and evaluate it on three benchmarks: Pix3D, ModelNet and NYU Depth V2. Extensive experiments show that our approach outperforms the previous state of the arts with a considerable margin.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑