Thermal-Det: 语言引导的跨模态蒸馏用于开放词汇热成像目标检测
Thermal-Det: Language-Guided Cross-Modal Distillation for Open-Vocabulary Thermal Object Detection
AI总结：
本文提出Thermal-Det，首个针对热成像的开放词汇检测器，通过合成数据集和跨模态蒸馏提升热成像目标检测性能，实验显示在公开基准上取得2-4%的AP提升。
摘要：
现有开放词汇检测器专注于RGB图像，难以泛化到热成像领域，其中低纹理和 emissivity 变化挑战基于RGB的语义。我们提出Thermal-Det，首个基于大语言模型（LLM）的开放词汇检测器，专为热图像设计。为实现大规模训练，我们通过将GroundingCap-1M转换到热域并过滤描述来生成合成数据集，得到超过一百万热对齐样本，包含边界框、接地文本和详细描述。Thermal-Det联合优化检测、描述和跨模态蒸馏目标。一个冻结的RGB教师模型为配对但未标注的RGB-热数据提供几何和语义伪监督，转移开放词汇知识而无需手动标注。模型进一步采用热文本对齐头进行文本校准，并采用模态融合的跨注意力模块进行双模态推理。不同于先前的领域适应方法，检测器完全微调以内化热对比模式，同时保持语言对齐。在公开基准上的实验显示，Thermal-Det在现有开放词汇检测器上取得一致的2-4% AP提升，为可扩展的语言驱动热感知奠定了坚实基础。
英文摘要：
Existing open-vocabulary detectors focus on RGB images and fail to generalize to thermal imagery, where low texture and emissivity variations challenge RGB-based semantics. We present Thermal-Det, the first large language model (LLM) supervised open-vocabulary detector tailored for thermal images. To enable large-scale training, we develop a synthetic dataset by converting GroundingCap-1M into the thermal domain and filtering captions to remove RGB-specific terms, yielding over one million thermally aligned samples with bounding boxes, grounding texts, and detailed captions. Thermal-Det jointly optimizes detection, captioning, and cross-modal distillation objectives. A frozen RGB teacher provides geometric and semantic pseudo-supervision for paired but unlabeled RGB-thermal data, transferring open-vocabulary knowledge without manual annotation. The model further employs a Thermal-Text Alignment Head for text calibration and a Modality-Fused Cross-Attention module for dual-modality reasoning. Unlike prior domain-adaptation methods, the detector is fully fine-tuned to internalize thermal contrast patterns while preserving language alignment. Experiments on public benchmarks show consistent 2-4% AP gains over existing open-vocabulary detectors, establishing a strong foundation for scalable, language-driven thermal perception.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
