Vision-Language Models seamlessly discriminate among arbitrary semantic categories, yet they still suffer from poor generalization when presented with challenging examples. For this reason, Episodic Test-Time Adaptation (TTA) strategies have recently emerged as powerful techniques to adapt VLMs in the presence of a single unlabeled image. The recent literature on TTA is dominated by the paradigm of prompt tuning by Marginal Entropy Minimization, which, relying on online backpropagation, inevitably slows down inference while increasing memory. In this work, we theoretically investigate the properties of this approach and unveil that a surprisingly strong TTA method lies dormant and hidden within it. We term this approach ZERO (TTA with "zero" temperature), whose design is both incredibly effective and frustratingly simple: augment N times, predict, retain the most confident predictions, and marginalize after setting the Softmax temperature to zero. Remarkably, ZERO requires a single batched forward pass through the vision encoder only and no backward passes. We thoroughly evaluate our approach following the experimental protocol established in the literature and show that ZERO largely surpasses or compares favorably w.r.t. the state-of-the-art while being almost 10x faster and 13x more memory-friendly than standard Test-Time Prompt Tuning. Thanks to its simplicity and comparatively negligible computation, ZERO can serve as a strong baseline for future work in this field. The code is available at https://github.com/FarinaMatteo/zero.
A Frustratingly Easy Improvement for Position Embeddings via Random Padding
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Center for Data Science, Peking University(北京大学数据科学中心);The MOE Key Laboratory of Computational Linguistics, Peking University(北京大学教育部计算语言学重点实验室)
Frustratingly Easy Majority Voting Using Length Conditioning
University of Modena and Reggio Emilia(摩德纳和雷焦艾米利亚大学);University of Pisa(比萨大学);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
Return of Frustratingly Easy Unsupervised Video Domain Adaptation
Magellan Technology Research Institute (MTRI)(马格纳技术研究所(MTRI));Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Nanyang Technological University(南洋理工大学)
Frustratingly Easy Feature Reconstruction for Out-of-Distribution Detection
MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China(MAIS与NLPR、自动化研究所、中国科学院、北京、中国);School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China(人工智能学院、中国科学院大学、北京、中国);Anhui University, Hefei, China(安徽大学、合肥、中国)
FIXED: Frustratingly Easy Domain Generalization with Mixup
Tsinghua University(清华大学);Microsoft(微软);Nanyang Technological University(南洋理工大学);University of North Carolina(北卡罗来纳大学);University of Chinese Academy of Sciences(中国科学院大学)
EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models
Jilin University(吉林大学);School of Computer Science, The University of Sydney(悉尼大学计算机科学学院);ByteDance(字节跳动);Institute of Translational Medicine, Shanghai Jiao Tong University(上海交通大学转化医学研究院)
Transfering Low-Frequency Features for Domain Adaptation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Development Research Institute of Guangzhou Smart City(广州智慧城市发展研究院)
PCaM: A Progressive Focus Attention-Based Information Fusion Method for Improving Vision Transformer Domain Adaptation
Westlake Institute for Advanced Studies(西湖研究学院);Centre for Artificial Intelligence(人工智能中心);State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)(多模态人工智能系统国家重点实验室);Institute of Automation(自动化研究所);Chinese Academy of Sciences (CASIA)(中国科学院);School of Artificial Intelligence(人工智能学院);University of Chinese Academy of Sciences (UCAS)(中国科学院大学);Alibaba Group(阿里巴巴集团);School of Computer Science(计算机学院);Engineering, the Faculty of Innovation Engineering, Macau University of Science(工程学院,创新工程 faculty,澳门科学大学);Microsoft(微软公司)
Few-Shot Adaptation of Pre-Trained Networks for Domain Shift
Institute for Infocomm Research, A*STAR(新加坡科技研究局信息通信研究所);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Centre for Frontier AI Research, A*STAR(新加坡科技研究局前沿人工智能研究中心)
Department of Automation, BNRist, Tsinghua University(清华大学自动化系、智能产业研究院(BNRist));Beijing Institute of Technology(北京理工大学);Carnegie Mellon University(卡内基梅隆大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院)