ProtoDCS: 向 robust 和 efficient 的 open-set 测试时间适应方法迈进：为视觉-语言模型提供稳健高效的开放集测试时间适应
ProtoDCS: Towards Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models
AI总结：
ProtoDCS通过概率高斯混合模型和证据驱动适应策略，解决视觉-语言模型在开放集测试时间适应中的稳健性和效率问题，提升模型准确性和异常检测能力。
摘要：
大规模视觉-语言模型（VLMs）表现出强大的零样本识别能力，但其在现实世界中的部署受到分布偏移的挑战。尽管测试时间适应（TTA）可以缓解这一问题，但现有的基于VLM的TTA方法在封闭集假设下运作，无法在开放集场景中处理包含协变量偏移的内分布（csID）和外分布（csOOD）数据。这导致了一个关键难题：模型必须区分未知的csOOD样本以避免干扰，同时同时适应已知的csID类以提高准确性。当前的开放集TTA（OSTTA）方法依赖于硬阈值分离和熵最小化适应。这些策略容易出错，常常错误分类模糊的csOOD样本并导致过度自信的预测，且其参数更新机制对VLMs来说计算成本过高。为了解决这些限制，我们提出了基于原型的双重检查分离（ProtoDCS），一种用于OSTTA的稳健框架，能够有效分离csID和csOOD样本，使VLMs能够安全高效地适应csID数据。我们的主要贡献是：（1）一种新的双重检查分离机制，采用概率高斯混合模型（GMM）验证来取代脆弱的阈值；（2）一种证据驱动的适应策略，利用不确定性感知损失和高效的原型级更新，缓解过度自信并减少计算开销。在CIFAR-10/100-C和Tiny-ImageNet-C上的广泛实验表明，ProtoDCS实现了最先进的性能，显著提升了已知类别的准确性和OOD检测指标。代码将在https://github.com/O-YangF/ProtoDCS上提供。
英文摘要：
Large-scale Vision-Language Models (VLMs) exhibit strong zero-shot recognition, yet their real-world deployment is challenged by distribution shifts. While Test-Time Adaptation (TTA) can mitigate this, existing VLM-based TTA methods operate under a closed-set assumption, failing in open-set scenarios where test streams contain both covariate-shifted in-distribution (csID) and out-of-distribution (csOOD) data. This leads to a critical difficulty: the model must discriminate unknown csOOD samples to avoid interference while simultaneously adapting to known csID classes for accuracy. Current open-set TTA (OSTTA) methods rely on hard thresholds for separation and entropy minimization for adaptation. These strategies are brittle, often misclassifying ambiguous csOOD samples and inducing overconfident predictions, and their parameter-update mechanism is computationally prohibitive for VLMs. To address these limitations, we propose Prototype-based Double-Check Separation (ProtoDCS), a robust framework for OSTTA that effectively separates csID and csOOD samples, enabling safe and efficient adaptation of VLMs to csID data. Our main contributions are: (1) a novel double-check separation mechanism employing probabilistic Gaussian Mixture Model (GMM) verification to replace brittle thresholding; and (2) an evidence-driven adaptation strategy utilizing uncertainty-aware loss and efficient prototype-level updates, mitigating overconfidence and reducing computational overhead. Extensive experiments on CIFAR-10/100-C and Tiny-ImageNet-C demonstrate that ProtoDCS achieves state-of-the-art performance, significantly boosting both known-class accuracy and OOD detection metrics. Code will be available at https://github.com/O-YangF/ProtoDCS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
