arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.07520cs.CV
收藏

IRSAM: 面向红外小目标检测的改进型Segment Anything Model

IRSAM: Advancing Segment Anything Model for Infrared Small Target Detection

Xidian University(西安电子科技大学);Chongqing University of Posts and Telecommunications(重庆邮电大学);The University of Sydney(悉尼大学)

Mingjin Zhang, Yuchun Wang, Jie Guo, Yunsong Li, Xinbo Gao, Jing Zhang

AI总结:

针对预训练SAM因域差距在红外小目标检测中表现不佳的问题,提出IRSAM模型,通过融入PMD模块的编码器与GAD解码器提升特征学习能力,在多个公开数据集上取得优于现有方法的性能。

摘要:

近期提出的Segment Anything Model(SAM,分割一切模型)是自然图像分割领域的一项重大进展,展现出强大的零样本性能,适用于各类下游图像分割任务。然而,由于自然图像与红外图像之间存在显著的域差距,直接将预训练SAM用于红外小目标检测(Infrared Small Target Detection, IRSTD)任务无法取得令人满意的性能。与可见光相机不同,热成像仪通过捕获红外辐射来呈现物体的温度分布,小目标通常在物体边界处表现出细微的温度过渡。为解决这一问题,我们提出了用于IRSTD的IRSAM模型,该模型改进了SAM的编码器-解码器架构,以学习红外小目标更优的特征表示。具体而言,我们设计了一个基于Perona-Malik扩散(PMD)的模块,并将其融入SAM编码器的多个层级,以帮助编码器在抑制噪声的同时捕获关键的结构特征。此外,我们设计了粒度感知解码器(Granularity-Aware Decoder, GAD),用于融合来自编码器的多粒度特征,以捕获长距离建模中可能丢失的结构信息。在NUAA-SIRST、NUDT-SIRST和IRSTD-1K等公开数据集上开展的大量实验,验证了IRSAM的设计合理性,以及其相较于代表性的当前最优方法的显著优越性。源代码已公开于:github.com/IPIC-Lab/IRSAM。

英文摘要:

The recent Segment Anything Model (SAM) is a significant advancement in natural image segmentation, exhibiting potent zero-shot performance suitable for various downstream image segmentation tasks. However, directly utilizing the pretrained SAM for Infrared Small Target Detection (IRSTD) task falls short in achieving satisfying performance due to a notable domain gap between natural and infrared images. Unlike a visible light camera, a thermal imager reveals an object's temperature distribution by capturing infrared radiation. Small targets often show a subtle temperature transition at the object's boundaries. To address this issue, we propose the IRSAM model for IRSTD, which improves SAM's encoder-decoder architecture to learn better feature representation of infrared small objects. Specifically, we design a Perona-Malik diffusion (PMD)-based block and incorporate it into multiple levels of SAM's encoder to help it capture essential structural features while suppressing noise. Additionally, we devise a Granularity-Aware Decoder (GAD) to fuse the multi-granularity feature from the encoder to capture structural information that may be lost in long-distance modeling. Extensive experiments on the public datasets, including NUAA-SIRST, NUDT-SIRST, and IRSTD-1K, validate the design choice of IRSAM and its significant superiority over representative state-of-the-art methods. The source code are available at: github.com/IPIC-Lab/IRSAM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑