由重复物体恢复结构:从一堆物体进行的神经逆向图形学
Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects
AI总结:
提出SfD框架,利用单张图像中相同物体的重复性作为先验,联合估计6-DoF位姿并逆向推理几何、材质与光照,实现了优于现有方法的3D重建。
摘要:
我们的世界充满了相同的物体(例如,可乐罐,同一型号的汽车)。这些重复物体在被同时看到时,为我们有效进行3D推理提供了额外且强烈的线索。受此观察启发,我们引入了 Structure from Duplicates (SfD),这是一种新颖的逆向图形学框架,可从包含多个相同物体的单张图像中重建几何、材质和光照。SfD 首先识别图像中物体的多个实例,然后联合估计所有实例的 6-DoF 位姿。随后采用逆向图形学管线,在遵循各实例间共享几何和材质约束的同时,联合推理物体的形状、材质以及环境光照。我们的主要贡献包括:利用物体重复作为单图像逆向图形学的鲁棒先验,并提出一种对平面内旋转鲁棒的 Structure from Motion (SfM) 公式化方法,用于联合 6-DoF 物体位姿估计。通过利用单张图像中的多视角线索,SfD 生成了更真实和详细的 3D 重建,显著优于现有的单图像重建模型以及具有相似或更多观测次数的多视角重建方法。
英文摘要:
Our world is full of identical objects (\emphe.g., cans of coke, cars of same model). These duplicates, when seen together, provide additional and strong cues for us to effectively reason about 3D. Inspired by this observation, we introduce Structure from Duplicates (SfD), a novel inverse graphics framework that reconstructs geometry, material, and illumination from a single image containing multiple identical objects. SfD begins by identifying multiple instances of an object within an image, and then jointly estimates the 6DoF pose for all instances.An inverse graphics pipeline is subsequently employed to jointly reason about the shape, material of the object, and the environment light, while adhering to the shared geometry and material constraint across instances. Our primary contributions involve utilizing object duplicates as a robust prior for single-image inverse graphics and proposing an in-plane rotation-robust Structure from Motion (SfM) formulation for joint 6-DoF object pose estimation. By leveraging multi-view cues from a single image, SfD generates more realistic and detailed 3D reconstructions, significantly outperforming existing single image reconstruction models and multi-view reconstruction approaches with a similar or greater number of observations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。