arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.03591cs.CVcs.AI
收藏

ObPose:利用位姿实现3D中以物体为中心的场景推理与生成

ObPose: Leveraging Pose for Object-Centric Scene Inference and Generation in 3D

University of Oxford(牛津大学)

Yizhe Wu, Oiwi Parker Jones, Ingmar Posner

AI总结:

该研究提出无监督3D以物体为中心的模型ObPose,通过分解隐空间、引入位姿归纳偏置和体素化NeRF近似方法,在多个数据集的场景分割任务上超越现有最优水平,还支持场景生成与编辑。

摘要:

我们提出了ObPose,这是一种无监督的以物体为中心的推理与生成模型,可从RGB-D场景中学习具备3D结构的隐式表示。受2D表示学习领域现有研究的启发,ObPose采用分解式隐空间,分别编码物体的位置(where)与外观(what)。ObPose进一步将通过最小体积原理定义的物体位姿(即位置与朝向)作为一种新型归纳偏置,用于学习位置分量。为实现这一点,我们提出了一种高效的体素化近似方法,可直接从神经辐射场(NeRF)中恢复物体形状。由此,ObPose将每个场景建模为多个NeRF的组合,可丰富地表示单个物体。为评估所学习表示的质量,我们在YCB、MultiShapeNet和CLEVR数据集上对ObPose的无监督场景分割性能进行了定量评估,其性能大幅优于当前3D场景推理领域的最优模型(ObSuRF)。生成结果的定性展示表明,同一个ObPose模型既可以生成新场景,也能灵活编辑场景中的物体。这些能力再次印证了所学习隐表示的质量,以及解耦场景的位置与外观分量的优势。我们还通过消融实验验证了ObPose编码器的关键设计选择。

英文摘要:

We present ObPose, an unsupervised object-centric inference and generation model which learns 3D-structured latent representations from RGB-D scenes. Inspired by prior art in 2D representation learning, ObPose considers a factorised latent space, separately encoding object location (where) and appearance (what). ObPose further leverages an object's pose (i.e. location and orientation), defined via a minimum volume principle, as a novel inductive bias for learning the where component. To achieve this, we propose an efficient, voxelised approximation approach to recover the object shape directly from a neural radiance field (NeRF). As a consequence, ObPose models each scene as a composition of NeRFs, richly representing individual objects. To evaluate the quality of the learned representations, ObPose is evaluated quantitatively on the YCB, MultiShapeNet, and CLEVR datatasets for unsupervised scene segmentation, outperforming the current state-of-the-art in 3D scene inference (ObSuRF) by a significant margin. Generative results provide qualitative demonstration that the same ObPose model can both generate novel scenes and flexibly edit the objects in them. These capacities again reflect the quality of the learned latents and the benefits of disentangling the where and what components of a scene. Key design choices made in the ObPose encoder are validated with ablations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑