arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.04789cs.CV
收藏

从单视图合成数据学习三维形状流形

Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data

Qianyu Feng, Yawei Luo, Keyang Luo, Yi Yang

摘要:

在现实世界中推断物体的立体结构是一个具有挑战性的实际任务。为深度模型配备这种能力通常需要大量的3D监督,这是很难获得的。很有希望的是,我们可以简单地从合成数据中受益,其中成对的地面实况很容易获得。然而,考虑到不同的纹理,形状和上下文,域间隙是不平凡的。为了克服这些困难,我们提出了一个视觉感知自适应网络的单视图三维重建,被称为VPAN。为了将该模型推广到真实场景,我们提出了几个方面:(1)外观:从单个视图中视觉上合并空间结构,以增强表示的表现力;(2)投射:通过跨模态语义对比映射将2D图像特征感知对齐到3D形状先验;(3)模具:通过将嵌入转换为所需的流形来重建目标的立体形状。在多个基准上的实验表明,该方法在通过单视图从合成数据学习三维形状流形方面是有效的和鲁棒的。所提出的方法在Pix 3D数据集上的性能优于最新技术,IoU为0.292,CD为0.108,在Pascal 3D+上达到IoU为0.329,CD为0.104。

英文摘要:

Inferring the stereo structure of objects in the real world is a challenging yet practical task. To equip deep models with this ability usually requires abundant 3D supervision which is hard to acquire. It is promising that we can simply benefit from synthetic data, where pairwise ground-truth is easy to access. Nevertheless, the domain gaps are nontrivial considering the variant texture, shape and context. To overcome these difficulties, we propose a Visio-Perceptual Adaptive Network for single-view 3D reconstruction, dubbed VPAN. To generalize the model towards a real scenario, we propose to fulfill several aspects: (1) Look: visually incorporate spatial structure from the single view to enhance the expressiveness of representation; (2) Cast: perceptually align the 2D image features to the 3D shape priors with cross-modal semantic contrastive mapping; (3) Mold: reconstruct stereo-shape of target by transforming embeddings into the desired manifold. Extensive experiments on several benchmarks demonstrate the effectiveness and robustness of the proposed method in learning the 3D shape manifold from synthetic data via a single-view. The proposed method outperforms state-of-the-arts on Pix3D dataset with IoU 0.292 and CD 0.108, and reaches IoU 0.329 and CD 0.104 on Pascal 3D+.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑