arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.08271cs.CVcs.AI
收藏

大视觉语言模型的高效提示调优用于细粒度船舶分类

Efficient Prompt Tuning of Large Vision-Language Model for Fine-Grained Ship Classification

College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机科学与技术学院);College of Physics and Information Engineering, Fuzhou University(福州大学物理与信息工程学院);School of Computer Science, Wuhan University(武汉大学计算机学院);College of Computer, National University of Defense Technology(国防科技大学计算机学院)

Long Lan, Fengxiang Wang, Xiangtao Zheng, Zengmao Wang, Xinwang Liu

AI总结:

针对遥感细粒度船舶分类中类间相似性高、标注数据有限的问题,提出分层多粒度提示调优技术,整合遥感船舶先验知识,引入FGSCM-52数据集,提升对未见船舶类别的分类准确性。

摘要:

遥感中的细粒度船舶分类(RS-FGSC)由于类间高度相似性和标注数据有限,对传统监督分类方法的有效性构成显著挑战。近年来,大型预训练视觉语言模型(VLMs)在少样本或零样本学习中展现出令人印象深刻的能力,尤其在理解图像内容方面。本研究深入探索利用VLMs的潜力,以提高对未见船舶类别的分类准确性,这在因成本或隐私限制导致数据受限的场景中具有重要意义。直接微调VLMs用于RS-FGSC常面临过拟合已见类别的挑战,导致对未见类别的泛化效果不佳,这凸显了区分复杂背景和捕捉独特船舶特征的难度。为解决这些问题,我们引入一种新颖的提示调优技术,采用分层、多粒度的提示设计。我们的方法通过从一个小型可训练网络学习的偏置项整合遥感船舶先验知识。该策略在提高模型区分复杂背景和学习判别性船舶特征能力的同时,增强了其泛化能力。此外,我们通过引入综合数据集FGSCM-52为该领域做出贡献,该数据集显著扩展了现有数据集,为较不常见的船舶类别提供了更广泛的数据和详细标注。广泛的实验评估表明,我们提出的方法优于当前最先进的技术。源代码将公开提供。

英文摘要:

Fine-grained ship classification in remote sensing (RS-FGSC) poses a significant challenge due to the high similarity between classes and the limited availability of labeled data, limiting the effectiveness of traditional supervised classification methods. Recent advancements in large pre-trained Vision-Language Models (VLMs) have demonstrated impressive capabilities in few-shot or zero-shot learning, particularly in understanding image content. This study delves into harnessing the potential of VLMs to enhance classification accuracy for unseen ship categories, which holds considerable significance in scenarios with restricted data due to cost or privacy constraints. Directly fine-tuning VLMs for RS-FGSC often encounters the challenge of overfitting the seen classes, resulting in suboptimal generalization to unseen classes, which highlights the difficulty in differentiating complex backgrounds and capturing distinct ship features. To address these issues, we introduce a novel prompt tuning technique that employs a hierarchical, multi-granularity prompt design. Our approach integrates remote sensing ship priors through bias terms, learned from a small trainable network. This strategy enhances the model's generalization capabilities while improving its ability to discern intricate backgrounds and learn discriminative ship features. Furthermore, we contribute to the field by introducing a comprehensive dataset, FGSCM-52, significantly expanding existing datasets with more extensive data and detailed annotations for less common ship classes. Extensive experimental evaluations demonstrate the superiority of our proposed method over current state-of-the-art techniques. The source code will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑