arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2507.00849cs.CV

UAVD-Mamba:用于多模态无人机检测的可变形令牌融合视觉Mamba

UAVD-Mamba: Deformable Token Fusion Vision Mamba for Multimodal UAV Detection

College of Mechanical and Vehicle Engineering, Hunan University(湖南大学机械与车辆工程学院);School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院)

Wei Li, Jiaman Tang, Yang Li, Beihao Xia, Ligang Tan, Hongmao Qin

AI总结:

本文提出UAVD-Mamba,一种基于Mamba架构的多模态无人机检测框架,通过可变形令牌融合、多尺度特征提取及交叉注意力机制,在DroneVehicle数据集上较基线OAFA方法mAP提升3.6%。

摘要:

无人机(UAV)目标检测已广泛应用于交通管理、农业、应急救援等领域。然而,它面临着遮挡、小目标尺寸和不规则形状等重大挑战。这些挑战凸显了对鲁棒且高效的多模态无人机目标检测方法的迫切需求。Mamba在多模态图像融合中已展现出相当大的潜力。基于此,我们提出了UAVD-Mamba,一个基于Mamba架构的多模态无人机目标检测框架。为了提高几何适应性,我们提出了可变形令牌Mamba模块(DTMB),通过将来自可变形卷积的自适应图像块与来自普通卷积的普通图像块相结合来生成可变形令牌,这些令牌作为Mamba模块的输入。为了优化多模态特征互补性,我们为RGB和红外(IR)模态分别设计了两个独立的DTMB,两个DTMB的输出被整合到Mamba模块中进行特征提取,并整合到融合Mamba模块中进行特征融合。此外,为了提高多尺度目标检测能力,尤其是针对小目标,我们在不同尺度上堆叠了四个DTMB以生成多尺度特征表示,然后将其送入用于Mamba的检测颈部(DNM)。DNM模块受YOLO系列启发,对YOLOv11的SPPF和C3K2进行了修改,以更好地处理多尺度特征。特别地,我们在DTMB之前采用交叉增强空间注意力,并在融合Mamba模块之后采用交叉通道注意力,以提取更具判别力的特征。在DroneVehicle数据集上的实验结果表明,我们的方法在mAP指标上比基线OAFA方法高出3.6%。代码将在https://github.com/GreatPlum-hnu/UAVD-Mamba.git发布。

英文摘要:

Unmanned Aerial Vehicle (UAV) object detection has been widely used in traffic management, agriculture, emergency rescue, etc. However, it faces significant challenges, including occlusions, small object sizes, and irregular shapes. These challenges highlight the necessity for a robust and efficient multimodal UAV object detection method. Mamba has demonstrated considerable potential in multimodal image fusion. Leveraging this, we propose UAVD-Mamba, a multimodal UAV object detection framework based on Mamba architectures. To improve geometric adaptability, we propose the Deformable Token Mamba Block (DTMB) to generate deformable tokens by incorporating adaptive patches from deformable convolutions alongside normal patches from normal convolutions, which serve as the inputs to the Mamba Block. To optimize the multimodal feature complementarity, we design two separate DTMBs for the RGB and infrared (IR) modalities, with the outputs from both DTMBs integrated into the Mamba Block for feature extraction and into the Fusion Mamba Block for feature fusion. Additionally, to improve multiscale object detection, especially for small objects, we stack four DTMBs at different scales to produce multiscale feature representations, which are then sent to the Detection Neck for Mamba (DNM). The DNM module, inspired by the YOLO series, includes modifications to the SPPF and C3K2 of YOLOv11 to better handle the multiscale features. In particular, we employ cross-enhanced spatial attention before the DTMB and cross-channel attention after the Fusion Mamba Block to extract more discriminative features. Experimental results on the DroneVehicle dataset show that our method outperforms the baseline OAFA method by 3.6% in the mAP metric. Codes will be released at https://github.com/GreatPlum-hnu/UAVD-Mamba.git.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑