Learning generalized representations from limited training samples is crucial for applying deep neural networks in low-resource scenarios. Recently, methods based on Contrastive Language-Image Pre-training (CLIP) have exhibited promising performance in few-shot adaptation tasks. To avoid catastrophic forgetting and overfitting caused by few-shot fine-tuning, existing works usually freeze the parameters of CLIP pre-trained on large-scale datasets, overlooking the possibility that some parameters might not be suitable for downstream tasks. To this end, we revisit CLIP's visual encoder with a specific focus on its distinctive attention pooling layer, which performs a spatial weighted-sum of the dense feature maps. Given that dense feature maps contain meaningful semantic information, and different semantics hold varying importance for diverse downstream tasks (such as prioritizing semantics like ears and eyes in pet classification tasks rather than side mirrors), using the same weighted-sum operation for dense features across different few-shot tasks might not be appropriate. Hence, we propose fine-tuning the parameters of the attention pooling layer during the training process to encourage the model to focus on task-specific semantics. In the inference process, we perform residual blending between the features pooled by the fine-tuned and the original attention pooling layers to incorporate both the few-shot knowledge and the pre-trained CLIP's prior knowledge. We term this method as Semantic-Aware FinE-tuning (SAFE). SAFE is effective in enhancing the conventional few-shot CLIP and is compatible with the existing adapter approach (termed SAFE-A).
CLIP-FSAC++: Few-Shot Anomaly Classification with Anomaly Descriptor Based on CLIP
Xi’an Jiaotong University（西安交通大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；Xiamen University（厦门大学）；Shenzhen University（深圳大学）
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Harbin Engineering University（哈尔滨工程大学）；Key Laboratory of Advanced Marine Communication and Information Technology（先进海洋通信与信息技术重点实验室）；University of Toronto（多伦多大学）；Kanagawa University（神奈川大学）
CLIP3D-AD: Extending CLIP for 3D Few-Shot Anomaly Detection with Multi-View Images Generation
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人机混合增强智能全国重点实验室与人工智能与机器人研究所）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；Xiamen University（厦门大学）；Shenzhen University（深圳大学）
AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP Adaptation
School of Computer Science and Engineering（计算机科学与工程学院）；Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（大数据分析与处理广东省重点实验室）
Label Shift Aware Adaptation for Online Zero-shot Learning with Contrastive Language-Image Pre-Training (CLIP)
Australian National University（澳大利亚国立大学）；China North Vehicle Research Institute（中国北方车辆研究所）；The Hong Kong Polytechnic University（香港理工大学）；Griffith University（格里菲斯大学）；CSIRO（澳大利亚联邦科学与工业研究组织）；The University of Hong Kong（香港大学）
Explaining CLIP Zero-shot Predictions Through Concepts
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Dept. of Computer Eng., Middle East Technical University (METU)（中东技术大学计算机工程系）；Orbital；DTU Compute, Technical University of Denmark（丹麦技术大学DTU计算学院）；Dept. of Biology, University of Copenhagen（哥本哈根大学生物系）；LTCI, Télécom Paris, Institut Polytechnique de Paris（巴黎理工学院巴黎电信学院LTCI实验室）；Technical University of Munich (TUM)（慕尼黑工业大学）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；MCML；MDSI；Robotics & AI Center (ROMER), METU（中东技术大学机器人与人工智能中心）
fine-CLIP: Enhancing Zero-Shot Fine-Grained Surgical Action Recognition with Vision-Language Models
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；INSERM（法国国家健康与医学研究院）；ICube（ICube实验室）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）
MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognition
School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）
Enhancing Concept Localization in CLIP-based Concept Bottleneck Models
Unité d’Informatique et d’Ingénierie des Systèmes ENSTA Paris, Institut Polytechnique de Paris（ENSTA巴黎信息与系统工程单位，巴黎理工学院）；Department of Information Engineering and Computer Science University of Trento（信息工程与计算机科学系，特伦托大学）
Language-Aware Information Maximization for Transductive Few-Shot CLIP
École de Technologie Supérieure (ÉTS)（École de Technologie Supérieure）；Université Catholique de Louvain (UCLouvain)（Université Catholique de Louvain）；Université de Mons (UMons)（Université de Mons）
VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；CASI Vision Technology CO., LTD.（中智视觉科技有限公司）；University of Technology Sydney（悉尼科技大学）；Hangzhou Dianzi University（杭州电子科技大学）；Tsinghua University（清华大学）
Few-shot Action Recognition with Permutation-invariant Attention
Australian National University（澳大利亚国立大学）；University of Oxford（牛津大学）；Data61/CSIRO（Data61/联邦科学与工业研究组织）；Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）；The University of Hong Kong（香港大学）
TLAC: Two-stage LMM Augmented CLIP for Zero-Shot Classification
Information Technology University（信息科技大学）；University of Ontario Institute of Technology（Ontario Institute of Technology 大学）；Mohamed Bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）
