arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.00877cs.CV
收藏

S2R-DepthNet:学习可推广的深度特定结构表示

S2R-DepthNet: Learning a Generalizable Depth-specific Structural Representation

University of Science and Technology of China(中国科学技术大学);Microsoft Research Asia(微软亚洲研究院)

Xiaotian Chen, Yuwang Wang, Xuejin Chen, Wenjun Zeng

摘要:

人类可以从草图而不是真实图像中推断出场景的三维几何形状,这表明空间结构在理解场景深度方面起着基础作用。我们是第一个探索学习的深度特定的结构表示,它捕捉的基本特征深度估计,忽略不相关的风格信息。我们的S2 R-DepthNet(合成到真实深度网络)可以很好地直接推广到看不见的真实世界数据,即使它只在合成数据上训练。S2 R-DepthNet包括:a)结构提取(STE)模块,其通过将图像分解成域不变结构和域特定风格分量来从图像提取域不变结构表示,b)深度特定注意力(DSA)模块,其学习任务特定知识以抑制深度无关结构,以用于更好的深度估计和泛化,以及c)深度预测模块(DP),用于根据所述深度特定表示来预测深度。在不访问任何真实世界图像的情况下,我们的方法甚至优于使用目标域的真实世界图像进行训练的最先进的无监督领域自适应方法。此外,当使用少量标记的真实世界数据时,我们在半监督设置下实现了最先进的性能。代码和训练模型可在https://github.com/microsoft/S2R-DepthNet上获得。

英文摘要:

Human can infer the 3D geometry of a scene from a sketch instead of a realistic image, which indicates that the spatial structure plays a fundamental role in understanding the depth of scenes. We are the first to explore the learning of a depth-specific structural representation, which captures the essential feature for depth estimation and ignores irrelevant style information. Our S2R-DepthNet (Synthetic to Real DepthNet) can be well generalized to unseen real-world data directly even though it is only trained on synthetic data. S2R-DepthNet consists of: a) a Structure Extraction (STE) module which extracts a domaininvariant structural representation from an image by disentangling the image into domain-invariant structure and domain-specific style components, b) a Depth-specific Attention (DSA) module, which learns task-specific knowledge to suppress depth-irrelevant structures for better depth estimation and generalization, and c) a depth prediction module (DP) to predict depth from the depth-specific representation. Without access of any real-world images, our method even outperforms the state-of-the-art unsupervised domain adaptation methods which use real-world images of the target domain for training. In addition, when using a small amount of labeled real-world data, we achieve the state-ofthe-art performance under the semi-supervised setting. The code and trained models are available at https://github.com/microsoft/S2R-DepthNet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑