arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.13892cs.CV
收藏

SideRT:一种用于单图像深度估计的实时的纯Transformer结构

SideRT: A Real-time Pure Transformer Architecture for Single Image Depth Estimation

Meituan Group(美团集团)

Chang Shu, Ziming Chen, Lei Chen, Kuan Ma, Minghui Wang, Haibing Ren

摘要:

由于上下文建模对于从单个图像估计深度至关重要,因此研究人员投入了大量精力来获取全局上下文。许多全局操作是为传统的基于CNN的架构设计的,以克服卷积的局部性。注意力机制或最初设计用于捕获远程依赖关系的Transformers可能是更好的选择,但通常会使体系结构复杂化,并可能导致推理速度降低。在这项工作中,我们提出了一种称为SideRT的纯Transformer架构,可以实时的【形】实现出色的预测。为了更好地捕捉全局上下文,跨尺度注意(CSA)和多尺度细化(MSR)模块被设计为协同工作,以有效地融合不同尺度的特征。CSA模块侧重于融合语义相似度高的特征,而MSR模块旨在融合相应位置的特征。这两个模块包含一些没有卷积的可学习参数,基于这些参数构建了一个轻量级但有效的模型。这种架构实现了最先进的实时的【形】性能(51.3 FPS),并且在较小的主干Swin-T(83.1 FPS)上具有合理的性能下降,速度更快。此外,它的性能大大超过了以前的最先进水平,KITTI的AbsRel指标提高了6.9%,NYU提高了9.7%。据我们所知,这是第一个工作表明,基于变换的网络可以达到最先进的性能,实时的【形】在单一图像深度估计领域。代码将很快提供。

英文摘要:

Since context modeling is critical for estimating depth from a single image, researchers put tremendous effort into obtaining global context. Many global manipulations are designed for traditional CNN-based architectures to overcome the locality of convolutions. Attention mechanisms or transformers originally designed for capturing long-range dependencies might be a better choice, but usually complicates architectures and could lead to a decrease in inference speed. In this work, we propose a pure transformer architecture called SideRT that can attain excellent predictions in real-time. In order to capture better global context, Cross-Scale Attention (CSA) and Multi-Scale Refinement (MSR) modules are designed to work collaboratively to fuse features of different scales efficiently. CSA modules focus on fusing features of high semantic similarities, while MSR modules aim to fuse features at corresponding positions. These two modules contain a few learnable parameters without convolutions, based on which a lightweight yet effective model is built. This architecture achieves state-of-the-art performances in real-time (51.3 FPS) and becomes much faster with a reasonable performance drop on a smaller backbone Swin-T (83.1 FPS). Furthermore, its performance surpasses the previous state-of-the-art by a large margin, improving AbsRel metric 6.9% on KITTI and 9.7% on NYU. To the best of our knowledge, this is the first work to show that transformer-based networks can attain state-of-the-art performance in real-time in the single image depth estimation field. Code will be made available soon.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑