arXiv 2503.234532025-03-30cs.CV

用于遥感图像描述的语义-空间特征融合与动态图细化

Semantic-Spatial Feature Fusion with Dynamic Graph Refinement for Remote Sensing Image Captioning

School of Computer Science and Technology, Wuhan University of Science and Technology（武汉科技大学计算机科学与技术学院）；School of Information Science and Engineering, Wuhan University of Science and Technology（武汉科技大学信息科学与工程学院）

Maofu Liu, Jiahui Liu, Xiaokang Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出SFDR方法，通过结合SSFF模块整合多级语义空间特征与DGFR模块利用图注意力及动态加权细化节点关系，有效解决了遥感图像描述中文本信息互补不足及对象定位不准的问题，显著提升了描述质量。

摘要：

遥感图像描述旨在生成与遥感图像视觉特征紧密相关的语义准确描述。现有方法通常注重视觉特征的细粒度提取和全局信息的捕捉。然而，它们往往忽略了文本信息在增强视觉语义中的互补作用，且在精确定位与图像上下文最相关的对象时面临挑战。为应对这些挑战，本文提出了一种带有动态图细化的语义-空间特征融合（SFDR）方法，该方法集成了语义-空间特征融合（SSFF）和动态图特征细化（DGFR）模块。SSFF模块利用预训练的CLIP特征、网格特征和ROI特征，采用多层次特征表示策略来整合丰富的语义和空间信息。在DGFR模块中，图注意力网络捕捉特征节点之间的关系，而动态加权机制则优先处理与当前场景最相关的对象，并抑制不太重要的对象。因此，所提出的SFDR方法显著提升了生成描述的质量。在三个基准数据集上的实验结果证明了所提方法的有效性。源代码将在 https://github.com/zxk688}{https://github.com/zx688 上提供。

英文摘要：

Remote sensing image captioning aims to generate semantically accurate descriptions that are closely linked to the visual features of remote sensing images. Existing approaches typically emphasize fine-grained extraction of visual features and capturing global information. However, they often overlook the complementary role of textual information in enhancing visual semantics and face challenges in precisely locating objects that are most relevant to the image context. To address these challenges, this paper presents a semantic-spatial feature fusion with dynamic graph refinement (SFDR) method, which integrates the semantic-spatial feature fusion (SSFF) and dynamic graph feature refinement (DGFR) modules. The SSFF module utilizes a multi-level feature representation strategy by leveraging pre-trained CLIP features, grid features, and ROI features to integrate rich semantic and spatial information. In the DGFR module, a graph attention network captures the relationships between feature nodes, while a dynamic weighting mechanism prioritizes objects that are most relevant to the current scene and suppresses less significant ones. Therefore, the proposed SFDR method significantly enhances the quality of the generated descriptions. Experimental results on three benchmark datasets demonstrate the effectiveness of the proposed method. The source code will be available at https://github.com/zxk688}{https://github.com/zxk688.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。