CLIP在军用车辆分类中的遮挡鲁棒性
Occlusion Robustness of CLIP for Military Vehicle Classification
AI总结：
该研究针对CLIP在军事遮挡场景下鲁棒性不足的问题，基于自制18类军用车辆数据集，用NAUC评估不同CLIP变体的遮挡鲁棒性，得出四项关键结论，为CLIP的国防场景部署提供参考。
摘要：
诸如CLIP之类的视觉语言模型（VLM）通过在共享嵌入空间中对齐图像与文本实现零样本分类，在标注数据稀缺的国防应用场景中具备优势。然而，CLIP在存在部分遮挡、信噪比（SNR）降低的复杂军事环境下的鲁棒性仍未得到充分研究。我们构建了包含18类军用车辆的自定义数据集，研究不同CLIP变体对遮挡的鲁棒性，并采用归一化曲线下面积（NAUC）在不同遮挡比例下开展评估。研究得出四项关键结论：（1）基于Transformer的CLIP模型性能始终优于CNN；（2）细粒度的分散式遮挡对性能的削弱程度大于大面积连续遮挡；（3）尽管线性探测模型的准确率有所提升，但在遮挡比例约为35%时性能会急剧下降；（4）通过微调模型骨干网络，该性能骤降的阈值可提升至60%以上遮挡比例。这些结果凸显了训练过程中引入遮挡相关数据增强的重要性，也表明CLIP的实际部署还需进一步探索补丁级敏感性与架构弹性。
英文摘要：
Vision-language models (VLMs) like CLIP enable zero-shot classification by aligning images and text in a shared embedding space, offering advantages for defense applications with scarce labeled data. However, CLIP's robustness in challenging military environments, with partial occlusion and degraded signal-to-noise ratio (SNR), remains underexplored. We investigate CLIP variants' robustness to occlusion using a custom dataset of 18 military vehicle classes and evaluate using Normalized Area Under the Curve (NAUC) across occlusion percentages. Four key insights emerge: (1) Transformer-based CLIP models consistently outperform CNNs, (2) fine-grained, dispersed occlusions degrade performance more than larger contiguous occlusions, (3) despite improved accuracy, performance of linear-probed models sharply drops at around 35% occlusion, (4) by finetuning the model's backbone, this performance drop occurs at more than 60% occlusion. These results underscore the importance of occlusion-specific augmentations during training and the need for further exploration into patch-level sensitivity and architectural resilience for real-world deployment of CLIP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
