从坐标到候选区域：遥感多模态大语言模型中的时序变化定位与区域选择
From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs
浏览论文内容
中文总结 AI 辅助
针对遥感多模态大语言模型在时序变化定位中的坐标生成脆弱性问题，提出区域选择范式，通过文本条件区域提议和特殊标记编码，在多项任务上显著优于基线并提升视觉基础性能。
中文摘要 AI 辅助
遥感多模态大语言模型（RS-MLLMs）在卫星影像的场景理解和视觉问答方面取得了进展，但定位特定目标或变化区域仍然具有挑战性。现有方法依赖生成边界框坐标作为标记序列，这对于遥感中常见的小型、密集目标而言较为脆弱，且当需要同时定位多个目标时，错误率会越来越高。在这项工作中，我们提出了区域选择范式的遥感专用表述，该范式此前已在自然图像多模态大语言模型中探索过，并将其扩展到多图像序列上的时序变化定位。我们的框架采用文本条件区域提议模块，将每个候选区域编码为携带逐帧视觉特征的特殊标记，这些特征融合了空间和时间线索，并让大语言模型通过在其响应中选择区域标记来定位目标。我们构建了一个多任务训练和评估套件，涵盖单图像和多时相设置下的定位、指代表达、视觉基础（visual grounding）和理解任务。实验表明，我们的方法在时序变化定位上大幅优于坐标生成基线，同时提升了单图像视觉基础性能，并保持了具有竞争力的理解性能。Oracle分析分解了区域提议器和LLM选择器的贡献，提供了该框架独有的诊断性见解。我们的代码将在此https URL中提供。
英文摘要
Remote sensing multimodal large language models (RS-MLLMs) have advanced scene understanding and visual question answering over satellite imagery, yet localizing specific objects or changed regions remains challenging. Existing approaches rely on generating bounding box coordinates as token sequences, which is fragile for the small, densely packed objects common in remote sensing and increasingly error-prone when multiple targets must be localized simultaneously. In this work, we present an RS-specific formulation of the region selection paradigm, previously explored in natural-image MLLMs, and extend it to temporal change localization over multi-image sequences. Our framework employs a text-conditioned region proposal module, encodes each candidate as special tokens carrying per-frame visual features enriched with spatial and temporal cues, and lets the LLM localize targets by selecting region tokens in its response. We construct a multi-task training and evaluation suite spanning localization, referring expression, visual grounding, and understanding tasks across single-image and multi-temporal settings. Experiments show that our approach substantially outperforms coordinate-generation baselines on temporal change localization, while improving single-image visual grounding and maintaining competitive understanding performance. Oracle analysis decomposes the contributions of the region proposer and the LLM selector, providing diagnostic insight unique to this framework. Our code will be available at https://github.com/juwan-kr/RS-RegionSelect.
发表机构
- KAIST（韩国科学技术院）
机构由 AI 辅助整理，请以论文原文为准。
