ChangeChat：一种基于多模态指令调优的遥感变化分析交互式模型
ChangeChat: An Interactive Model for Remote Sensing Change Analysis via Multimodal Instruction Tuning
AI总结：
针对传统遥感变化检测缺上下文、现有变化字幕不支持交互查询的问题，提出首个专用双时相VLM ChangeChat，经多模态指令调优及自研ChangeChat-87k数据集训练，可处理多类复杂查询，性能媲美SOTA且远超GPT-4。
摘要：
遥感（RS）变化分析通过检测不同时间图像中的变化，对监测地球动态过程至关重要。传统变化检测擅长识别像素级变化，但缺乏对这些变化进行上下文关联的能力。尽管近期变化字幕技术的进展能提供变化的自然语言描述，但它们不支持交互式的、用户特定的查询。为解决这些局限，我们提出ChangeChat，这是首个专门为遥感变化分析设计的双时相视觉语言模型（VLM）。ChangeChat采用多模态指令调优，能够处理复杂查询，如变化字幕、特定类别量化和变化定位。为提升模型性能，我们构建了ChangeChat-87k数据集，该数据集结合基于规则的方法和GPT辅助技术生成。实验表明，ChangeChat为遥感变化分析提供了全面的交互式解决方案，在特定任务上性能与当前最优（SOTA）方法相当甚至更优，且显著超越最新的通用领域模型GPT-4。代码和预训练权重可在https://github.com/hanlinwu/ChangeChat获取。
英文摘要：
Remote sensing (RS) change analysis is vital for monitoring Earth's dynamic processes by detecting alterations in images over time. Traditional change detection excels at identifying pixel-level changes but lacks the ability to contextualize these alterations. While recent advancements in change captioning offer natural language descriptions of changes, they do not support interactive, user-specific queries. To address these limitations, we introduce ChangeChat, the first bitemporal vision-language model (VLM) designed specifically for RS change analysis. ChangeChat utilizes multimodal instruction tuning, allowing it to handle complex queries such as change captioning, category-specific quantification, and change localization. To enhance the model's performance, we developed the ChangeChat-87k dataset, which was generated using a combination of rule-based methods and GPT-assisted techniques. Experiments show that ChangeChat offers a comprehensive, interactive solution for RS change analysis, achieving performance comparable to or even better than state-of-the-art (SOTA) methods on specific tasks, and significantly surpassing the latest general-domain model, GPT-4. Code and pre-trained weights are available at https://github.com/hanlinwu/ChangeChat.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
