Adapting CLIP to vertical domains is typically approached by novel fine-tuning strategies or by continual pre-training (CPT) on large domain-specific datasets. Yet, data itself remains an underexplored factor in this process. We revisit this task from a data-centric perspective: Can effective data selection substitute for large-scale datasets in CPT? We introduce CHIPS (Curvature-aware Hybrid Influence in Projection Subspace), which assigns each image-text pair a utility score that integrates three complementary factors aligned with three goals: faithfulness via a curvature-aware and Newton-style alignment computed in CLIP's end-point subspace; scalability via an InfoNCE-aware curvature estimator with Johnson-Lindenstrauss (JL) sketching; and retention via a selection-aware relevance weight combined with learnability to balance target adaptation against general-domain preservation. We justify this design theoretically by proving a lower-bound guarantee on the proxy's correlation with full-parameter alignment and by characterizing the bias-variance trade-offs introduced by curvature mixing and JL sketching. We evaluate CHIPS empirically across various settings: 1) CHIPS attains state-of-the-art performance among selection baselines on 17 medical benchmarks, matches full-dataset CPT with 30% of the data, and outperforms half-dataset CPT using only 10%; 2) on 31 general-domain benchmarks, CHIPS yields the least performance drop under all retention ratios.
CLIPArTT: Adaptation of CLIP to New Domains at Test Time
LIVIA, ÉTS Montréal(LIVIA实验室,蒙特利尔工程学院);International Laboratory on Learning Systems (ILLS)(国际学习系统实验室);McGill(麦吉尔大学);ETS(蒙特利尔工程学院);MILA(蒙特利尔学习算法研究所);CNRS(法国国家科学研究中心);Université Paris-Saclay(巴黎萨克雷大学);CentraleSupélec(中央理工-高等电力学院)
FAA-CLIP: Federated Adversarial Adaptation of CLIP
School of Artificial Intelligence, Guilin University of Electronic Technology(桂林电子科技大学人工智能学院);College of Medicine, Taibah University(塔伊夫大学医学院);College of Computer Science and Engineering, Jeddah University(吉达大学计算机科学与工程学院);Ecole de Technologie Superieure(高等技术学院)
CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Survey
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));The Jilin University(吉林大学);Griffith University(格里菲斯大学);The Chinese University of Hong Kong(香港中文大学)
COSMIC: Clique-Oriented Semantic Multi-space Integration for Robust CLIP Test-Time Adaptation
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Shenzhen Technology University(深圳技术大学);University of Science and Technology of China(中国科学技术大学)
Lightweight Prompt-Guided CLIP Adaptation for Monocular Depth Estimation
School of Engineering Science, Simon Fraser University(西蒙弗雷泽大学工程科学学院);School of Information Science and Technology, Eastern Institute of Technology(东方理工学院信息科学与技术学院)
A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation
University of Science and Technology of China(中国科学技术大学);CRIPAC & MAIS, Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所CRIPAC与MAIS);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Nanjing University(南京大学)
Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks
Department of CSE, Southern University of Science and Technology(南方科技大学计算机科学与工程系);College of Mathematics and Statistics, Chongqing University(重庆大学数学与统计学院);ACS Laboratory, Huawei Technologies Co., Ltd.(华为技术有限公司ACS实验室)
On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMs
IDEAS Research Institute(IDEAS研究所);Warsaw University of Technology(华沙理工大学);Akces NCBR(NCBR Akces);NASK National Research Institute(NASK国家研究所);Computer Vision Center, Barcelona(巴塞罗那计算机视觉中心)
TriplePlay: Enhancing Federated Learning with CLIP for Non-IID Data and Resource Efficiency
School of Computing, Southern Illinois University(南伊利诺伊大学计算机学院);Security, Privacy and Intelligence for Edge Devices Laboratory (SPEED Lab)(边缘设备安全、隐私与智能实验室(SPEED Lab))