零样本多光谱学习：为遥感应用重新构想通用多模态 Gemini 2.5 模型
Zero-Shot Multi-Spectral Learning: Reimagining a Generalist Multimodal Gemini 2.5 Model for Remote Sensing Applications
AI总结：
本文提出一种免训练的零样本方法，将多光谱遥感数据适配到通用多模态模型 Gemini 2.5 的 RGB 视觉空间并注入领域指令，在土地覆盖与土地利用分类基准上取得显著性能提升，展示了通用模型处理专门遥感输入的潜力。
摘要：
多光谱影像在土地利用分类、环境监测和城市规划等多种遥感应用中发挥着至关重要的作用。这些影像之所以被广泛采用，是因为其额外的光谱波段与地面上的物理物质（如冰、水和植被）密切相关，从而能够实现更准确的识别；而且它们可从 Sentinel-2 和 Landsat 等任务中公开获取，进一步提升了其价值。目前，此类数据的自动分析主要由专门针对多光谱输入训练的机器学习模型来完成，这些模型的训练和维护成本高昂。此外，尽管这些额外输入为遥感提供了大量实用价值，但它们无法与强大的通用大型多模态模型一起使用——这些模型能够解决许多视觉问题，却无法理解专门的多光谱信号。为解决这一问题，我们提出了一种免训练方法，以纯零样本方式将新的多光谱数据作为输入引入仅在 RGB 输入上训练的通用多模态模型。我们的方法利用多模态模型对视觉空间的理解，提出将输入适配到该视觉空间，并将领域特定信息作为指令注入模型中。我们以 Gemini2.5 模型为例展示了这一思路，观察到该方法在流行的土地覆盖和土地利用分类遥感基准上取得了显著的零样本性能提升，并证明了 Gemini2.5 对新输入具有易适应性。这些结果凸显了处理非标准专门输入的地理空间专业人员可以轻松利用 Gemini2.5 等强大模型来加速工作的潜力，从而受益于其丰富的推理和上下文能力，并以专门的传感器数据为基础。
英文摘要：
Multi-spectral imagery plays a crucial role in diverse Remote Sensing applications including land-use classification, environmental monitoring and urban planning. These images are widely adopted because their additional spectral bands correlate strongly with physical materials on the ground, such as ice, water, and vegetation. This allows for more accurate identification, and their public availability from missions, such as Sentinel-2 and Landsat, only adds to their value. Currently, the automatic analysis of such data is predominantly managed through machine learning models specifically trained for multi-spectral input, which are costly to train and support. Furthermore, although providing a lot of utility for Remote Sensing, such additional inputs cannot be used with powerful generalist large multimodal models, which are capable of solving many visual problems, but are not able to understand specialized multi-spectral signals. To address this, we propose a training-free approach which introduces new multi-spectral data in a Zero-Shot-only mode, as inputs to generalist multimodal models, trained on RGB-only inputs. Our approach leverages the multimodal models' understanding of the visual space, and proposes to adapt to inputs to that space, and to inject domain-specific information as instructions into the model. We exemplify this idea with the Gemini2.5 model and observe strong Zero-Shot performance gains of the approach on popular Remote Sensing benchmarks for land cover and land use classification and demonstrate the easy adaptability of Gemini2.5 to new inputs. These results highlight the potential for geospatial professionals, working with non-standard specialized inputs, to easily leverage powerful multimodal models, such as Gemini2.5, to accelerate their work, benefiting from their rich reasoning and contextual capabilities, grounded in the specialized sensor data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
