CLIPArTT：测试时将CLIP适配到新领域
CLIPArTT: Adaptation of CLIP to New Domains at Test Time
AI总结：
针对CLIP在领域偏移下性能下降问题，提出全测试时适配方法CLIPArTT，通过推理时自动构建文本提示并聚合预测类别为伪标签转导分类，标准化VLMs的TTA基准，在多类数据集上动态提升性能。
摘要：
以CLIP为代表的预训练视觉语言模型（VLMs）在零样本分类任务中展现出卓越的适应性，无需额外训练。然而，在存在领域偏移时，其性能会下降。本研究提出CLIP测试时适配方法（CLIPArTT），这是一种针对CLIP的全测试时适配（TTA）方法，在推理过程中自动构建文本提示作为文本监督。我们的方法采用独特的微创文本提示调优过程，将多个预测类别聚合为单个新文本提示，作为伪标签以转导方式对输入重新分类。此外，我们率先在VLMs领域标准化TTA基准（如TENT）。研究结果表明，无需额外变换或新的可训练模块，CLIPArTT在非损坏数据集（如CIFAR-100）、损坏数据集（如CIFAR-100-C和ImageNet-C）以及合成数据集（如VisDA-C）上动态提升性能。本研究强调了通过新型测试时策略提高VLMs适应性的潜力，为在不同数据集和环境中实现稳健性能提供了见解。代码可在https://github.com/dosowiechi/CLIPArTT.git获取。
英文摘要：
Pre-trained vision-language models (VLMs), exemplified by CLIP, demonstrate remarkable adaptability across zero-shot classification tasks without additional training. However, their performance diminishes in the presence of domain shifts. In this study, we introduce CLIP Adaptation duRing Test-Time (CLIPArTT), a fully test-time adaptation (TTA) approach for CLIP, which involves automatic text prompts construction during inference for their use as text supervision. Our method employs a unique, minimally invasive text prompt tuning process, wherein multiple predicted classes are aggregated into a single new text prompt, used as \emph{pseudo label} to re-classify inputs in a transductive manner. Additionally, we pioneer the standardization of TTA benchmarks (e.g., TENT) in the realm of VLMs. Our findings demonstrate that, without requiring additional transformations nor new trainable modules, CLIPArTT enhances performance dynamically across non-corrupted datasets such as CIFAR-100, corrupted datasets like CIFAR-100-C and ImageNet-C, alongside synthetic datasets such as VisDA-C. This research underscores the potential for improving VLMs' adaptability through novel test-time strategies, offering insights for robust performance across varied datasets and environments. The code can be found at: https://github.com/dosowiechi/CLIPArTT.git
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
