Sector Patch Embedding:一种符合鱼眼图像畸变模式的嵌入模块
Sector Patch Embedding: An Embedding Module Conforming to The Distortion Pattern of Fisheye Image
摘要:
鱼眼相机在具有大视场(LFOV)的同时遭受图像失真。这一事实导致了一些鱼眼视觉任务的性能不佳。解决方法之一是对现有的鱼眼图像视觉算法进行优化。然而,大多数基于CNN的方法和基于Transformer的方法缺乏有效利用失真信息的能力。在这项工作中,我们提出了一种新的补丁嵌入方法称为扇区补丁嵌入(SPE),符合鱼眼图像的失真模式。在此基础上,我们提出了一个基于ImageNet-1 K的合成鱼眼数据集,并在此数据集上研究了几种Transformer模型的性能。SPE使ViT和PVT的前1分类准确率分别提高了0.75%和2.8%。实验结果表明,本文提出的扇形块嵌入方法能更好地感知鱼眼图像的畸变,并能更好地提取特征。我们的方法可以很容易地采用到其他基于transformer的模型。源代码位于https://github.com/IN2-ViAUn/Sector-Patch-Embedding。
英文摘要:
Fisheye cameras suffer from image distortion while having a large field of view(LFOV). And this fact leads to poor performance on some fisheye vision tasks. One of the solutions is to optimize the current vision algorithm for fisheye images. However, most of the CNN-based methods and the Transformer-based methods lack the capability of leveraging distortion information efficiently. In this work, we propose a novel patch embedding method called Sector Patch Embedding(SPE), conforming to the distortion pattern of the fisheye image. Furthermore, we put forward a synthetic fisheye dataset based on the ImageNet-1K and explore the performance of several Transformer models on the dataset. The classification top-1 accuracy of ViT and PVT is improved by 0.75% and 2.8% with SPE respectively. The experiments show that the proposed sector patch embedding method can better perceive distortion and extract features on the fisheye images. Our method can be easily adopted to other Transformer-based models. Source code is at https://github.com/IN2-ViAUn/Sector-Patch-Embedding.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。