ViDA：用于持续测试时适应的稳态视觉域适配器
ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation
AI总结：
针对持续测试时适应中的错误累积和灾难性遗忘问题，提出视觉域适配器ViDA及稳态知识分配策略HKA，通过结合高秩和低秩特征分别适应域分布和维持共享知识，在分类与分割任务上达到SOTA。
摘要：
由于现实世界的机器系统在非平稳环境中运行，持续测试时适应（CTTA）任务被提出，以使预训练模型适应不断变化的目标域。近期，现有方法主要关注基于模型的适应，旨在利用自训练方式提取目标域知识。然而，伪标签可能带有噪声，且在动态数据分布下更新的模型参数不可靠，导致在持续适应过程中出现错误累积和灾难性遗忘。为应对这些挑战并保持模型可塑性，我们为CTTA设计了视觉域适配器（ViDA），明确处理域特定和域共享知识。具体而言，我们首先全面探索了具有可训练高秩或低秩嵌入空间的适配器的不同域表示。然后我们将ViDA注入预训练模型，利用高秩和低秩特征分别适应当前域分布和维持持续的域共享知识。为更有效地利用低秩和高秩ViDA，我们进一步提出稳态知识分配（HKA）策略，自适应地结合来自各ViDA的不同知识。在四个广泛使用的基准上的大量实验表明，我们提出的方法在分类和分割CTTA任务中均取得了最先进的性能。注意，我们的方法可被视为大规模模型的新颖迁移范式，在不断变化分布的适应中交付了有前景的结果。项目页面：https://sites.google.com/view/iclr2024-vida/home。
英文摘要：
Since real-world machine systems are running in non-stationary environments, Continual Test-Time Adaptation (CTTA) task is proposed to adapt the pre-trained model to continually changing target domains. Recently, existing methods mainly focus on model-based adaptation, which aims to leverage a self-training manner to extract the target domain knowledge. However, pseudo labels can be noisy and the updated model parameters are unreliable under dynamic data distributions, leading to error accumulation and catastrophic forgetting in the continual adaptation process. To tackle these challenges and maintain the model plasticity, we design a Visual Domain Adapter (ViDA) for CTTA, explicitly handling both domain-specific and domain-shared knowledge. Specifically, we first comprehensively explore the different domain representations of the adapters with trainable high-rank or low-rank embedding spaces. Then we inject ViDAs into the pre-trained model, which leverages high-rank and low-rank features to adapt the current domain distribution and maintain the continual domain-shared knowledge, respectively. To exploit the low-rank and high-rank ViDAs more effectively, we further propose a Homeostatic Knowledge Allotment (HKA) strategy, which adaptively combines different knowledge from each ViDA. Extensive experiments conducted on four widely used benchmarks demonstrate that our proposed method achieves state-of-the-art performance in both classification and segmentation CTTA tasks. Note that, our method can be regarded as a novel transfer paradigm for large-scale models, delivering promising results in adaptation to continually changing distributions. Project page: https://sites.google.com/view/iclr2024-vida/home.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
