增强遥感视觉-语言模型以实现零样本场景分类
Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classification
AI总结：
针对遥感视觉-语言模型分块独立预测导致上下文丢失的问题，本文提出无监督、低成本的RS-TransCLIP转导推理方法，融合文本提示预测与图像块亲和关系，在10个数据集上显著提升零样本场景分类准确率。
摘要：
用于遥感的视觉-语言模型凭借其大规模预训练已展现出良好的应用前景。然而，它们在零样本场景分类方法中的传统用法仍需将大图像划分为图像块并进行独立预测，即归纳推理，因此忽视了宝贵的上下文信息，限制了其有效性。我们的方法通过利用基于文本提示的初始预测以及来自图像编码器的图像块亲和关系来解决这一问题，以转导推理增强零样本能力，整个过程无需监督，且计算成本很小。在10个遥感数据集上使用最先进的视觉-语言模型进行的实验表明，相较于归纳式零样本分类，准确率取得了显著提升。我们的源代码已在Github公开发布：https://github.com/elkhouryk/RS-TransCLIP
英文摘要：
Vision-Language Models for remote sensing have shown promising uses thanks to their extensive pretraining. However, their conventional usage in zero-shot scene classification methods still involves dividing large images into patches and making independent predictions, i.e., inductive inference, thereby limiting their effectiveness by ignoring valuable contextual information. Our approach tackles this issue by utilizing initial predictions based on text prompting and patch affinity relationships from the image encoder to enhance zero-shot capabilities through transductive inference, all without the need for supervision and at a minor computational cost. Experiments on 10 remote sensing datasets with state-of-the-art Vision-Language Models demonstrate significant accuracy improvements over inductive zero-shot classification. Our source code is publicly available on Github: https://github.com/elkhouryk/RS-TransCLIP
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
