arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.03499cs.CVcs.AI
收藏

通过几何空间聚合器学习连续深度表示

Learning Continuous Depth Representation via Geometric Spatial Aggregator

Shanghai Jiao Tong University(上海交通大学)

Xiaohang Wang, Xuanhong Chen, Bingbing Ni, Zhengyan Tong, Hang Wang

摘要:

深度图超分辨率(DSR)是三维计算机视觉的一项基本任务。虽然在这种情况下,任意尺度DSR是更现实的设置,但先前的方法主要遭受低效的实数尺度上采样的问题。为了明确解决这个问题,我们提出了一种新的连续深度表示DSR。这种表示的核心是我们提出的几何空间聚合器(GSA),它利用了由任意上采样目标网格调制的距离场,通过它将几何信息显式地引入到特征聚合和目标生成中。此外,砖与GSA,我们提出了一个transformer风格的骨干称为GeoDSR,它拥有一个原则性的方法来构建本地坐标和高分辨率输出结果之间的函数映射,赋予我们的模型与任意形状变换的优势,准备帮助不同的缩放需求。在标准深度图基准上的广泛实验结果,例如,NYU v2已经证明,与现有技术相比,所提出的框架在任意尺度深度图超分辨率中实现了显著的恢复增益。https://github.com/nana01219/GeoDSR

英文摘要:

Depth map super-resolution (DSR) has been a fundamental task for 3D computer vision. While arbitrary scale DSR is a more realistic setting in this scenario, previous approaches predominantly suffer from the issue of inefficient real-numbered scale upsampling. To explicitly address this issue, we propose a novel continuous depth representation for DSR. The heart of this representation is our proposed Geometric Spatial Aggregator (GSA), which exploits a distance field modulated by arbitrarily upsampled target gridding, through which the geometric information is explicitly introduced into feature aggregation and target generation. Furthermore, bricking with GSA, we present a transformer-style backbone named GeoDSR, which possesses a principled way to construct the functional mapping between local coordinates and the high-resolution output results, empowering our model with the advantage of arbitrary shape transformation ready to help diverse zooming demand. Extensive experimental results on standard depth map benchmarks, e.g., NYU v2, have demonstrated that the proposed framework achieves significant restoration gain in arbitrary scale depth map super-resolution compared with the prior art. Our codes are available at https://github.com/nana01219/GeoDSR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑