基于Transformer交叉注意力的极端3D图像旋转估计
Estimating Extreme 3D Image Rotation with Transformer Cross-Attention
AI总结:
针对大角度及极端3D图像旋转估计问题,提出基于CNN特征与Transformer编码器的交叉注意力方法,替代传统4D相关体,在常用数据集上取得SOTA精度并公开代码。
摘要:
大角度及极端图像旋转估计在多个计算机视觉领域中发挥着关键作用,这些场景下旋转后的图像之间仅存在有限视场重叠,甚至无重叠视场。现有方法采用卷积神经网络计算4D相关体,以估计图像对之间的相对旋转。本研究提出一种基于交叉注意力的方法,利用CNN特征图与Transformer编码器,计算图像对激活图之间的交叉注意力——经证实,该交叉注意力是对以往研究中所用4D相关体的改进等效形式。在所提方法中,更高的注意力分数对应编码了旋转视觉线索的图像区域。该方法可端到端训练,且通过简单的回归损失进行优化。实验表明,在常用的图像旋转数据集和基准测试上,该方法的性能优于当前最先进的方案,并在这些数据集上创下了新的最高精度。我们已将代码公开。
英文摘要:
The estimation of large and extreme image rotation plays a key role in multiple computer vision domains, where the rotated images are related by a limited or a non-overlapping field of view. Contemporary approaches apply convolutional neural networks to compute a 4D correlation volume to estimate the relative rotation between image pairs. In this work, we propose a cross-attention-based approach that utilizes CNN feature maps and a Transformer-Encoder, to compute the cross-attention between the activation maps of the image pairs, which is shown to be an improved equivalent of the 4D correlation volume, used in previous works. In the suggested approach, higher attention scores are associated with image regions that encode visual cues of rotation. Our approach is end-to-end trainable and optimizes a simple regression loss. It is experimentally shown to outperform contemporary state-of-the-art schemes when applied to commonly used image rotation datasets and benchmarks, and establishes a new state-of-the-art accuracy on these datasets. We make our code publicly available.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。