GeoPixel：遥感中的像素级接地大型多模态模型
GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing
AI总结：
提出GeoPixel，首个支持像素级接地的端到端高分辨率遥感LMM，通过生成交错掩码实现细粒度视觉感知，并构建数据集GeoPixelD，在单/多目标分割任务上超越现有模型。
摘要：
大型多模态模型（LMMs）的最新进展已将细粒度接地视为视觉理解和对话的关键因素。然而，这种表示在LMMs中的优势仅限于自然图像领域，这些模型在遥感（RS）方面表现不佳。高分辨率遥感影像中独特的俯视视角、尺度变化和小目标的存在，对区域级理解提出了独特挑战。此外，LMMs在遥感中接地对话能力的发展受到缺乏粒度细、遥感领域特定的接地数据的阻碍。针对这些限制，我们提出了GeoPixel——首个支持像素级接地的端到端高分辨率遥感LMM。该能力通过在对话中生成交错掩码，实现细粒度的视觉感知。GeoPixel支持任意纵横比下高达4K高清分辨率，非常适合高精度遥感图像分析。为了支持遥感影像中的接地对话生成（GCG），我们通过半自动化流程策划了一个视觉接地数据集GeoPixelD，该流程利用集合标记提示和针对遥感数据定制的空间先验，系统地控制数据生成过程。GeoPixel在像素级理解方面表现出卓越性能，在单目标和多目标分割任务中均超越了现有LMMs。我们的方法消融研究验证了整体架构中每个组件的有效性。我们的代码和数据将公开发布。
英文摘要：
Recent advances in large multimodal models (LMMs) have recognized fine-grained grounding as an imperative factor of visual understanding and dialogue. However, the benefits of such representation in LMMs are limited to the natural image domain, and these models perform poorly for remote sensing (RS). The distinct overhead viewpoint, scale variation, and presence of small objects in high-resolution RS imagery present a unique challenge in region-level comprehension. Moreover, the development of the grounding conversation capability of LMMs within RS is hindered by the lack of granular, RS domain-specific grounded data. Addressing these limitations, we propose GeoPixel - the first end-to-end high resolution RS-LMM that supports pixel-level grounding. This capability allows fine-grained visual perception by generating interleaved masks in conversation. GeoPixel supports up to 4K HD resolution in any aspect ratio, ideal for high-precision RS image analysis. To support the grounded conversation generation (GCG) in RS imagery, we curate a visually grounded dataset GeoPixelD through a semi-automated pipeline that utilizes set-of-marks prompting and spatial priors tailored for RS data to methodically control the data generation process. GeoPixel demonstrates superior performance in pixel-level comprehension, surpassing existing LMMs in both single-target and multi-target segmentation tasks. Our methodological ablation studies validate the effectiveness of each component in the overall architecture. Our code and data will be publicly released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
