发表机构
TNO（荷兰应用科学研究组织）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对未见无人机域红外车辆检测的红外数据稀缺问题，采用生成式RGB到红外图像转换方法，利用ControlNet的Stable Diffusion 3.5生成合成红外数据，显著提升了RF-DETR检测器在Kust4K和VTUAV数据集上的检测性能。
AI 中文摘要
当真实世界数据稀缺时，合成训练数据对视觉AI的开发至关重要，热红外（IR）无人机车辆检测便是如此。尽管丰富的无人机RGB图像推动了用于数据增强的RGB到红外转换，但不可观测的热特征（如发动机热量）使学习可迁移映射变得具有挑战性。本研究探讨现代生成转换器能否克服这种跨模态差距，以提升未见无人机目标域的红外车辆检测性能。转换器在配对的RGB-红外源数据集上训练，应用于保留的目标数据集的RGB训练图像以生成合成红外数据。评估的方法包括监督GAN、基于ControlNet的扩散模型，以及通过LoRA进行的基础模型编辑。所得合成红外图像用于训练RF-DETR车辆检测器，在五个航空数据集的未见红外目标测试分割上进行评估，其中Kust4K和VTUAV作为目标域。合成红外数据始终优于RGB和灰度基线。带有ControlNet的Stable Diffusion 3.5取得最佳结果，与仅在源域红外数据上训练的模型相比，在Kust4K上的mAP从50.8提升至60.1，在VTUAV上从25.6提升至38.4。通过多个种子（+1.1 mAP）和提示变化（+3.3 mAP）增加输出多样性，在VTUAV上进一步提升性能。尽管与真实目标红外数据仍存在性能差距，但生成式RGB到红外转换有效缓解了红外数据稀缺问题，提升了跨域航空车辆检测性能。
英文摘要
Synthetic training data is crucial for developing vision AI when real-world data is scarce, as in thermal infrared (IR) aerial vehicle detection. While abundant UAV RGB imagery motivates RGB-to-IR translation for data augmentation, unobservable thermal traits (e.g., engine heat) make learning transferable mappings challenging. This work investigates whether modern generative translators can overcome this cross-modal gap to improve infrared vehicle detection on unseen UAV target domains. Translators are trained on paired RGB-IR source datasets and applied to RGB training images from held-out target datasets to generate synthetic IR data. Evaluated methods include supervised GANs, ControlNet-based diffusion models, and foundation-model editing via LoRA. The resulting synthetic IR imagery is used to train RF-DETR vehicle detectors, which are evaluated on unseen IR target test splits across five aerial datasets, with Kust4K and VTUAV serving as target domains. Synthetic IR consistently outperforms RGB and grayscale baselines. Stable Diffusion 3.5 with ControlNet yields the best results, improving mAP from 50.8 to 60.1 on Kust4K and from 25.6 to 38.4 on VTUAV compared to models trained only on source-domain IR data. Increasing output diversity via multiple seeds (+1.1 mAP) and prompt variations (+3.3 mAP) provides additional gains on VTUAV. Although a performance gap to real target IR data remains, generative RGB-to-IR translation effectively mitigates IR data scarcity and improves cross-domain aerial vehicle detection.
CommentsSubmitted to SPIE Sensors + Imaging 2026
