变化到描述：一种用于遥感变化描述的注意力网络
Changes to Captions: An Attentive Network for Remote Sensing Change Captioning
AI总结：
本文提出Chg2Cap注意力网络，结合Siamese CNN、分层自注意力和Transformer，用于双时相遥感图像变化描述，并在两个数据集上验证其有效性。
摘要：
近年来，前沿研究越来越关注利用自然语言处理（NLP）技术直接学习和分析遥感图像。准确描述多时相遥感图像中发生的变化，对于地理空间理解和土地规划正变得日益重要。与自然图像变化描述任务不同，遥感变化描述旨在捕捉最显著的变化，而不受光照、季节效应和复杂土地覆盖等多种影响因素干扰。在本研究中，我们强调了准确描述遥感图像变化的重要性，并对自然图像、合成图像与遥感图像的变化描述任务进行了比较。为解决生成准确描述所面临的挑战，我们提出了一种面向双时相遥感图像的注意力变化到描述网络，简称Chg2Cap。该网络包含三个主要组件：1）一个基于Siamese CNN的特征提取器，用于为每对图像收集高层表示；2）一个注意力解码器，其中包括一个分层自注意力模块以定位与变化相关的特征，以及一个残差模块以生成图像嵌入；3）一个基于Transformer的描述生成器，用于将图像嵌入与词嵌入之间的关系解码为文本描述。所提出的Chg2Cap网络在两个具有代表性的遥感数据集上进行了评估，并给出了全面的实验分析。代码和预训练模型将在https://github.com/ShizhenChang/Chg2Cap上公开。
英文摘要：
In recent years, advanced research has focused on the direct learning and analysis of remote sensing images using natural language processing (NLP) techniques. The ability to accurately describe changes occurring in multi-temporal remote sensing images is becoming increasingly important for geospatial understanding and land planning. Unlike natural image change captioning tasks, remote sensing change captioning aims to capture the most significant changes, irrespective of various influential factors such as illumination, seasonal effects, and complex land covers. In this study, we highlight the significance of accurately describing changes in remote sensing images and present a comparison of the change captioning task for natural and synthetic images and remote sensing images. To address the challenge of generating accurate captions, we propose an attentive changes-to-captions network, called Chg2Cap for short, for bi-temporal remote sensing images. The network comprises three main components: 1) a Siamese CNN-based feature extractor to collect high-level representations for each image pair; 2) an attentive decoder that includes a hierarchical self-attention block to locate change-related features and a residual block to generate the image embedding; and 3) a transformer-based caption generator to decode the relationship between the image embedding and the word embedding into a description. The proposed Chg2Cap network is evaluated on two representative remote sensing datasets, and a comprehensive experimental analysis is provided. The code and pre-trained models will be available online at https://github.com/ShizhenChang/Chg2Cap.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
