ProtoHGF-Net：用于RGBT目标检测的模态内校准原型超图融合网络
ProtoHGF-Net: Prototype HyperGraph Fusion with Intra-modal Calibration for RGBT Object Detection
浏览论文内容
中文总结 AI 辅助
针对现有RGBT目标检测方法密集跨模态交互易引入背景干扰的问题，提出ProtoHGF-Net框架，通过原型级语义交互与教师掩码校准蒸馏技术，在三个公开数据集上取得最优性能。
中文摘要 AI 辅助
RGB-热成像（RGBT）目标检测通过利用可见光纹理与热线索的互补优势，可在复杂场景中实现鲁棒感知。然而，现有方法主要依赖全分辨率特征上的密集跨模态交互，这不可避免地引入背景干扰，阻碍与目标相关表征的学习。本文提出原型超图融合网络（Prototype HyperGraph Fusion Network，ProtoHGF-Net），将跨模态融合重新定义为原型级语义交互而非密集跨模态交互范式。具体而言，我们设计原型超图融合，在紧凑的原型级语义空间中执行跨模态交互，该设计支持与目标相关原型间更具选择性的融合。为支撑该原型级融合，我们提出教师掩码校准蒸馏，其利用模态特定教师与目标感知掩码在融合前校准模态特征，该策略抑制背景主导响应并生成更聚焦目标的特征。在DroneVehicle、DVTOD和FLIR数据集上的大量实验表明，ProtoHGF-Net分别达到85.9%的$mAP_{50}$、88.2%的$mAP_{50}$和79.1%的$mAP_{50}$，实现了最优性能。我们的代码可在GitHub获取。
英文摘要
RGB-Thermal (RGBT) object detection enables robust perception in complex scenes by leveraging the complementary strengths of visible textures and thermal cues. However, existing methods mainly rely on dense cross-modal interactions over full-resolution features, which inevitably introduce background interference and hinder the learning of target-relevant representations. In this paper, we propose the Prototype HyperGraph Fusion Network (ProtoHGF-Net), a novel framework that redefines cross-modal fusion as prototype-level semantic interaction rather than the dense cross-modal interaction paradigm. Specifically, we design Prototype HyperGraph Fusion to perform cross-modal interaction in a compact prototype-level semantic space. This design enables more selective fusion among target-relevant prototypes. To support this prototype-level fusion, we propose Teacher-Mask Calibration Distillation, which calibrates modality features before fusion using modality-specific teachers and target-aware masks. This strategy suppresses backgrou- nd-dominant responses and produces more target-focused features. Extensive experiments on DroneVehicle, DVTOD, and FLIR demonstrate that ProtoHGF-Net achieves state-of-the-art performance with 85.9\% $mAP_{50}$, 88.2\% $mAP_{50}$, and 79.1\% $mAP_{50}$, respectively. Our code is available at \href{https://github.com/ZiMo-Chen/ProtoHGF}{GitHub}.
发表机构
- Zhejiang Normal University（浙江师范大学）
- National University of Defense Technology（国防科技大学）
机构由 AI 辅助整理，请以论文原文为准。
