The advancements in generative modeling, particularly the advent of diffusion models, have sparked a fundamental question: how can these models be effectively used for discriminative tasks? In this work, we find that generative models can be great test-time adapters for discriminative models. Our method, Diffusion-TTA, adapts pre-trained discriminative models such as image classifiers, segmenters and depth predictors, to each unlabelled example in the test set using generative feedback from a diffusion model. We achieve this by modulating the conditioning of the diffusion model using the output of the discriminative model. We then maximize the image likelihood objective by backpropagating the gradients to discriminative model's parameters. We show Diffusion-TTA significantly enhances the accuracy of various large-scale pre-trained discriminative models, such as, ImageNet classifiers, CLIP models, image pixel labellers and image depth predictors. Diffusion-TTA outperforms existing test-time adaptation methods, including TTT-MAE and TENT, and particularly shines in online adaptation setups, where the discriminative model is continually adapted to each example in the test set. We provide access to code, results, and visualizations on our website: https://diffusion-tta.github.io/.
Benchmarking Test-Time Adaptation against Distribution Shifts in Image Classification
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Science and Technology of China（中国科学技术大学）
The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models
University of Science and Technology of China（中国科学技术大学）；NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Princeton University（普林斯顿大学）；Penn State University（宾夕法尼亚州立大学）
Recently, test-time adaptation (TTA) has been proposed as a promising solution for addressing distribution shifts. It allows a base model to adapt to an unforeseen distribution during inference by leveraging the information from the batch of (unlabeled) test data. However, we uncover a novel security vulnerability of TTA based on the insight that predictions on benign samples can be impacted by malicious samples in the same batch. To exploit this vulnerability, we propose Distribution Invading Attack (DIA), which injects a small fraction of malicious data into the test batch. DIA causes models…
South China University of Technology（华南理工大学）；Tencent AI Lab（腾讯人工智能实验室）；National University of Singapore（新加坡国立大学）；Pazhou Laboratory（琶洲实验室）
Test-time adaptation (TTA) has shown to be effective at tackling distribution shifts between training and testing data by adapting a given model on test samples. However, the online model updating of TTA may be unstable and this is often a key obstacle preventing existing TTA methods from being deployed in the real world. Specifically, TTA may fail to improve or even harm the model performance when test data have: 1) mixed distribution shifts, 2) small batch sizes, and 3) online imbalanced label distribution shifts, which are quite common in practice. In this paper, we investigate the unstable…
A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
Center for Research on Intelligent Perception and Computing（智能感知与计算研究中心）；State Key Laboratory of Multimodal Artificial Intelligence Systems（多模态人工智能系统国家重点实验室）；CASIA（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Nanjing University（南京大学）
TTAQ: Towards Stable Post-training Quantization in Continuous Domain Adaptation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Inspur Yunzhou Industrial Internet Co., Ltd.（浪潮云洲工业互联网有限公司）
Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation
Artificial Intelligence Research Institute, Shenzhen MSU-BIT University and Guangdong-Hong Kong-Macao Joint Laboratory for Emotional Intelligence and Pervasive Computing（人工智能研究院、深圳MSU-BIT大学及粤港澳大湾区情感智能与普适计算联合实验室）；School of Software Engineering, South China University of Technology（华南理工大学软件学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Test-time adaptation (TTA) is an emerging paradigm that addresses distributional shifts between training and testing phases without additional data acquisition or labeling cost; only unlabeled test data streams are used for continual model adaptation. Previous TTA schemes assume that the test samples are independent and identically distributed (i.i.d.), even though they are often temporally correlated (non-i.i.d.) in application scenarios, e.g., autonomous driving. We discover that most existing TTA methods fail dramatically under such scenarios. Motivated by this, we present a new test-time a…
Qualcomm AI Research（高通人工智能研究院）；Qualcomm Technologies, Inc.（高通技术公司）
We consider the problem of improving the human instance segmentation mask quality for a given test image using keypoints estimation. We compare two alternative approaches. The first approach is a test-time adaptation (TTA) method, where we allow test-time modification of the segmentation network's weights using a single unlabeled test image. In this approach, we do not assume test-time access to the labeled source dataset. More specifically, our TTA method consists of using the keypoints estimates as pseudo labels and backpropagating them to adjust the backbone weights. The second approach is …
Carnegie Mellon University（卡内基梅隆大学）；Bosch Center for AI（博世人工智能中心）
Test-time adaptation (TTA) refers to adapting neural networks to distribution shifts, with access to only the unlabeled test samples from the new domain at test-time. Prior TTA methods optimize over unsupervised objectives such as the entropy of model predictions in TENT [Wang et al., 2021], but it is unclear what exactly makes a good TTA loss. In this paper, we start by presenting a surprising phenomenon: if we attempt to meta-learn the best possible TTA loss over a wide class of functions, then we recover a function that is remarkably similar to (a temperature-scaled version of) the softmax-…
AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs
The University of Melbourne, Australia（墨尔本大学）；Nokia Bell Labs, UK（诺基亚贝尔实验室）；The University of Auckland, New Zealand（奥克兰大学）；University of Cambridge, UK（剑桥大学）；The University of New South Wales, Australia（新南威尔士大学）；University of Glasgow, UK（格拉斯哥大学）
ETAGE: Enhanced Test Time Adaptation with Integrated Entropy and Gradient Norms for Robust Model Performance
Concordia University（康考迪亚大学）；Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）；UNSW Sydney（新南威尔士大学悉尼分校）；Concordia Institute of Information Systems Engineering（康考迪亚信息系统工程学院）；The Graduate School of Biomedical Engineering（生物医学工程研究生院）；School of Biomedical Engineering（生物医学工程学院）
Sample-Aware Test-Time Adaptation for Medical Image-to-Image Translation
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma（人工智能与计算机系统单位，工程系，罗马生物医学大学）；Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University（诊断与干预系，辐射物理，生物医学工程，乌梅大学）
POEM: Explore Unexplored Reliable Samples to Enhance Test-Time Adaptation
School of Electronic and Information Engineering & Guangdong Provincial Key Laboratory of Industrial Intelligent Inspection Technology, Foshan University, Foshan, China（电子与信息工程学院及广东工业智能检测技术省级重点实验室，佛山大学，佛山，中国）；College of Computing and Data Science, Nanyang Technological University (NTU), Singapore（计算与数据科学学院，南洋理工大学（NTU），新加坡）
