Diffusion-RSCC：用于遥感图像变化描述的扩散概率模型
Diffusion-RSCC: Diffusion Probabilistic Model for Change Captioning in Remote Sensing Images
AI总结：
针对遥感双时相图像变化描述中像素差异干扰与跨模态生成难题，提出条件扩散概率模型Diffusion-RSCC，并在LEVIR-CC上验证其性能优势。
摘要：
遥感图像变化描述（RSICC）旨在生成类人语言，以描述双时相遥感图像对之间的语义变化。它为环境动态和土地管理提供了宝贵洞见。与传统的变化描述任务不同，RSICC不仅涉及跨不同模态检索相关信息并生成流畅的描述，还涉及缓解像素级差异对地形变化定位的影响。由长时间跨度导致的像素问题会降低所生成描述的准确性。受扩散模型显著生成能力的启发，我们提出一种用于RSICC的概率扩散模型，以解决上述问题。在训练过程中，我们构建了一个以跨模态特征为条件的噪声预测器，用于在马尔可夫链下学习从真实描述分布到标准高斯分布的分布。同时，在反向过程中为噪声预测器设计了跨模态融合和堆叠自注意力模块。在测试阶段，训练良好的噪声预测器有助于估计分布的均值，并逐步生成变化描述。在LEVIR-CC数据集上进行的大量实验证明了我们的Diffusion-RSCC及其各个组件的有效性。定量结果显示，在传统指标和新增指标上，其性能均优于现有方法。代码和材料将在https://github.com/Fay-Y/Diffusion-RSCC上在线提供。
英文摘要：
Remote sensing image change captioning (RSICC) aims at generating human-like language to describe the semantic changes between bi-temporal remote sensing image pairs. It provides valuable insights into environmental dynamics and land management. Unlike conventional change captioning task, RSICC involves not only retrieving relevant information across different modalities and generating fluent captions, but also mitigating the impact of pixel-level differences on terrain change localization. The pixel problem due to long time span decreases the accuracy of generated caption. Inspired by the remarkable generative power of diffusion model, we propose a probabilistic diffusion model for RSICC to solve the aforementioned problems. In training process, we construct a noise predictor conditioned on cross modal features to learn the distribution from the real caption distribution to the standard Gaussian distribution under the Markov chain. Meanwhile, a cross-mode fusion and a stacking self-attention module are designed for noise predictor in the reverse process. In testing phase, the well-trained noise predictor helps to estimate the mean value of the distribution and generate change captions step by step. Extensive experiments on the LEVIR-CC dataset demonstrate the effectiveness of our Diffusion-RSCC and its individual components. The quantitative results showcase superior performance over existing methods across both traditional and newly augmented metrics. The code and materials will be available online at https://github.com/Fay-Y/Diffusion-RSCC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
