基于CLIP与提示学习的小样本遥感图像场景分类
Few-Shot Remote Sensing Image Scene Classification with CLIP and Prompt Learning
AI总结：
针对遥感图像场景分类标注数据稀缺、CLIP直接应用存在领域差距的问题，系统探索多种提示学习方法用于小样本场景分类，验证其性能优于基线，且自调节约束提示跨域表现最优。
摘要：
遥感应用愈发依赖深度学习实现场景分类，但由于标注数据稀缺，且不同地理区域、传感器领域的标注成本高昂，其性能往往受到限制。近期的视觉-语言模型（如CLIP）通过对齐视觉与文本模态，大规模学习可迁移的表征，展现出应用潜力，但由于存在显著的领域差距，且需要针对任务进行语义适配，直接将其应用于遥感领域的效果仍不理想。\n为解决这一关键挑战，我们系统探索了提示学习作为一种轻量高效的适配策略，用于小样本遥感图像场景分类任务。我们评估了多种代表性方法，包括Context Optimization（上下文优化）、Conditional Context Optimization（条件上下文优化）、Multi-modal Prompt Learning（多模态提示学习）以及Prompting with Self-Regulating Constraints（自调节约束提示）。这些方法体现了互补的设计理念：从静态上下文优化，到用于提升泛化性的条件提示，再到实现视觉-语言联合适配的多模态提示，以及用于稳定学习且避免遗忘的语义正则化提示。\n我们将这些提示学习方法与两种标准基线进行了基准对比：采用手工设计提示的零样本CLIP，以及基于冻结CLIP特征训练的线性探测头。通过在多个标准遥感数据集上开展大量实验（包括跨数据集泛化测试），我们证明在小样本场景下，提示学习的表现始终优于两种基线。值得注意的是，Prompting with Self-Regulating Constraints取得了最鲁棒的跨域性能。我们的研究结果表明，提示学习是弥合卫星与航空影像领域差距的一种可扩展且高效的解决方案，为该领域的未来研究奠定了坚实基础。
英文摘要：
Remote sensing applications increasingly rely on deep learning for scene classification. However, their performance is often constrained by the scarcity of labeled data and the high cost of annotation across diverse geographic and sensor domains. While recent vision-language models like CLIP have shown promise by learning transferable representations at scale by aligning visual and textual modalities, their direct application to remote sensing remains suboptimal due to significant domain gaps and the need for task-specific semantic adaptation. To address this critical challenge, we systematically explore prompt learning as a lightweight and efficient adaptation strategy for few-shot remote sensing image scene classification. We evaluate several representative methods, including Context Optimization, Conditional Context Optimization, Multi-modal Prompt Learning, and Prompting with Self-Regulating Constraints. These approaches reflect complementary design philosophies: from static context optimization to conditional prompts for enhanced generalization, multi-modal prompts for joint vision-language adaptation, and semantically regularized prompts for stable learning without forgetting. We benchmark these prompt-learning methods against two standard baselines: zero-shot CLIP with hand-crafted prompts and a linear probe trained on frozen CLIP features. Through extensive experiments on multiple benchmark remote sensing datasets, including cross-dataset generalization tests, we demonstrate that prompt learning consistently outperforms both baselines in few-shot scenarios. Notably, Prompting with Self-Regulating Constraints achieves the most robust cross-domain performance. Our findings underscore prompt learning as a scalable and efficient solution for bridging the domain gap in satellite and aerial imagery, providing a strong foundation for future research in this field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
