面向遥感变化描述的像素级变化检测伪标签学习
Pixel-Level Change Detection Pseudo-Label Learning for Remote Sensing Change Captioning
AI总结：
针对遥感变化描述任务复杂场景性能差、缺乏像素级变化检测标签的问题，提出带伪标签监督的辅助变化检测分支和语义融合增强模块，实现了当前最优性能。
摘要：
现有的遥感图像变化描述（RSICC）方法在简单场景中表现良好，但在复杂场景中性能较差。这一局限主要源于模型在区分和定位变化方面的视觉能力受限。鉴于变化检测（CD）与RSICC任务之间的内在关联，我们认为像素级CD对于通过语言描述图像间的差异具有重要意义。遗憾的是，当前的RSICC数据集缺乏现成的像素级CD标签。为弥补这一不足，我们利用在现有CD数据集上训练的模型来生成CD伪标签。我们提出了一种带有辅助CD分支的创新网络，该分支由伪标签监督。此外，我们提出了语义融合增强（SFA）模块，用于融合CD分支提取的特征信息，从而促进对变化的细致描述。实验表明，我们的方法达到了当前最优（state-of-the-art）性能，并验证了学习像素级CD伪标签对变化描述具有显著贡献。我们的代码将发布于：https://github.com/Chen-Yang-Liu/Pix4Cap
英文摘要：
The existing methods for Remote Sensing Image Change Captioning (RSICC) perform well in simple scenes but exhibit poorer performance in complex scenes. This limitation is primarily attributed to the model's constrained visual ability to distinguish and locate changes. Acknowledging the inherent correlation between change detection (CD) and RSICC tasks, we believe pixel-level CD is significant for describing the differences between images through language. Regrettably, the current RSICC dataset lacks readily available pixel-level CD labels. To address this deficiency, we leverage a model trained on existing CD datasets to derive CD pseudo-labels. We propose an innovative network with an auxiliary CD branch, supervised by pseudo-labels. Furthermore, a semantic fusion augment (SFA) module is proposed to fuse the feature information extracted by the CD branch, thereby facilitating the nuanced description of changes. Experiments demonstrate that our method achieves state-of-the-art performance and validate that learning pixel-level CD pseudo-labels significantly contributes to change captioning. Our code will be available at: https://github.com/Chen-Yang-Liu/Pix4Cap
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
