arXiv 2509.231052025-12-30cs.CV

迈向遥感中全面交互变化理解：一个大规模数据集和双粒度增强VLM

Towards Comprehensive Interactive Change Understanding in Remote Sensing: A Large-scale Dataset and Dual-granularity Enhanced VLM

Research Center for Space Computing System, Zhejiang Lab（浙江省实验室空间计算系统研究中心）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；School of Cyber Science and Engineering, Zhengzhou University（郑州大学网络科学与工程学院）；Liaoning University of Technology（辽宁科技学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University（武汉测绘遥感与信息工程国家重点实验室（LIESMARS））；College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；College of Intelligent Robotics and Advanced Manufacturing, Fudan University（复旦大学智能机器人与先进制造学院）

Junxiao Xue, Quan Deng, Xuecheng Wu, Kelu Yao, Xinyi Yin, Fei Yu, Wei Zhou, Yanfei Zhong, Yang Liu, Dingkang Yang

备注：: Junxiao Xue, Quan Deng, and Xuecheng Wu deserve equal contributions

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出ChangeIMTI数据集和ChangeVG模型，通过双粒度增强方法提升遥感图像变化理解的准确性和交互性。

摘要：

遥感变化理解（RSCU）对于分析遥感图像和理解人类活动如何影响环境至关重要。然而，现有数据集在多样化的变化描述、计数和定位任务中缺乏深度理解和交互。为解决这些差距，我们构建了ChangeIMTI，一个新的大规模交互多任务指令数据集，涵盖四个互补的任务，包括变化描述、二元变化分类、变化计数和变化定位。在此新数据集的基础上，我们进一步设计了一个新型的视觉引导视觉语言模型（ChangeVG），具有双粒度意识，用于双时间遥感图像（即同一区域在不同时间的两幅遥感图像）。引入的视觉引导模块是一个双分支架构，协同结合细粒度空间特征提取与高层语义摘要。这些丰富表示进一步作为辅助提示，指导大型视觉语言模型（VLMs）（例如Qwen2.5-VL-7B）进行指令微调，从而促进层次化的跨模态学习。我们广泛地在四个任务上进行实验，以展示我们方法的优越性。值得注意的是，在变化描述任务中，我们的方法在综合S*m指标上比最强的方法Semantic-CC高出1.39分，该指标整合了语义相似性和描述准确性，以提供对变化描述的总体评估。此外，我们还进行了一系列消融研究，以检验我们方法的关键组成部分。本工作的源代码和相关数据已公开在Github上。

英文摘要：

Remote sensing change understanding (RSCU) is essential for analyzing remote sensing images and understanding how human activities affect the environment. However, existing datasets lack deep understanding and interactions in the diverse change captioning, counting, and localization tasks. To tackle these gaps, we construct ChangeIMTI, a new large-scale interactive multi-task instruction dataset that encompasses four complementary tasks including change captioning, binary change classification, change counting, and change localization. Building upon this new dataset, we further design a novel vision-guided vision-language model (ChangeVG) with dual-granularity awareness for bi-temporal remote sensing images (i.e., two remote sensing images of the same area at different times). The introduced vision-guided module is a dual-branch architecture that synergistically combines fine-grained spatial feature extraction with high-level semantic summarization. These enriched representations further serve as the auxiliary prompts to guide large vision-language models (VLMs) (e.g., Qwen2.5-VL-7B) during instruction tuning, thereby facilitating the hierarchical cross-modal learning. We extensively conduct experiments across four tasks to demonstrate the superiority of our approach. Remarkably, on the change captioning task, our method outperforms the strongest method Semantic-CC by 1.39 points on the comprehensive S*m metric, which integrates the semantic similarity and descriptive accuracy to provide an overall evaluation of change caption. Moreover, we also perform a series of ablation studies to examine the critical components of our method. The source code and associated data for this work are publicly available at Github.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。