解决潜在扩散模型中RGB到SWIR图像翻译的细节瓶颈
Addressing Detail Bottlenecks in Latent Diffusion for RGB-to-SWIR Image Translation
AI总结：
针对潜在扩散模型在RGB到SWIR图像翻译中丢失空间细节的问题，提出源条件自编码器和可学习引导编码器两种轻量级改进，在驾驶场景下将检测mAP提升至2倍，小目标提升3.4倍，并达到最优FID。
摘要：
潜在扩散模型（LDM）能够高效地进行图像到图像的翻译，但在压缩过程中丢弃了精细的空间细节，从而降低了下游感知任务的性能。我们识别出两个瓶颈：自编码器（丢失空间信息）和条件路径（通过朴素下采样进一步退化源信号）。我们提出了两种轻量级、与骨干网络无关的修复方法：源条件自编码器（SCAE），通过跳跃连接将高分辨率源特征注入解码器；以及可学习引导编码器（LGE），用学习到的条件信号替代朴素下采样。在驾驶场景的RGB到SWIR翻译任务上，使用两种去噪骨干网络（U-Net和DiT）进行评估，我们的方法在潜在扩散基线基础上将检测mAP提升了高达2倍，小目标（COCO-small，<32^2像素^2）上提升高达3.4倍，同时达到了最先进的FID。我们进一步表明FID与检测性能相关性较差，从而激励多轴评估。结果零样本泛化到公开的RASMD基准。我们将公开发布带有标注的测试数据、所有检查点和训练代码。
英文摘要：
Latent diffusion models (LDMs) enable efficient image-to-image translation but discard fine spatial details during compression, degrading downstream perception tasks. We identify two bottlenecks: the autoencoder, which loses spatial information, and the conditioning pathway, which further degrades the source signal through naive downsampling. We propose two lightweight, backbone-agnostic fixes: a Source-Conditioned Autoencoder (SCAE) that injects high-resolution source features into the decoder via skip connections, and a Learnable Guidance Encoder (LGE) that replaces naive downsampling with a learned conditioning signal. Evaluated on RGB-to-SWIR translation for driving scenes with two denoiser backbones (U-Net and DiT), our approach improves detection mAP by up to 2x over the latent diffusion baseline, with up to 3.4x gains on small objects (COCO-small, <32^2 px^2), while achieving state-of-the-art FID. We further show that FID and detection performance are poorly correlated, motivating multi-axis evaluation. Results generalise zero-shot to the public RASMD benchmark. We will publicly release test data with annotations, all checkpoints, and training code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
