基于双分布估计的零样本噪声测试时自适应方法
Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs
AI总结：
针对噪声测试时自适应中分布外异常值问题，提出双分布估计框架，通过无训练的高斯分布建模实现零样本鲁棒自适应。
摘要：
虽然测试时自适应（TTA）使视觉语言模型无需昂贵重训练即可适应，但在实际应用中极易受到分布外（OOD）异常值的影响。这一差异催生了噪声TTA（NTTA），即在线任务，在最大化分布内（ID）分类准确率的同时，动态过滤噪声OOD样本。现有的零样本NTTA方法通常依赖测试时判别训练，导致过度自信的错误分类和推理效率显著下降。为解决这些限制，我们提出名为双分布估计（DDE）的新框架，将零样本NTTA范式从实例级学习转变为无训练的高斯分布建模。DDE包含两个新颖模块：正特征分布估计（PFDE）和负标签分布估计（NLDE）。PFDE显式建模类别包含和排除高斯分布，以形成校准的对比分数，稳健地提升ID准确率。同时，NLDE通过显式建模负标签分布以挖掘高判别性标签，有效缓解虚假相关性，从而改进OOD识别。大量实验表明，在大规模ImageNet基准上，DDE在调和平均准确率上提升了3.70%，并将OOD检测的FPR95降低了6.20%，同时确保高度可扩展和高效的在线推理。此外，DDE是零样本且无需训练的，在数据稀缺场景下展现出显著的鲁棒性。代码可在该https URL获取。
英文摘要：
While test-time adaptation (TTA) empowers vision-language models to adapt without costly retraining, it remains highly vulnerable to out-of-distribution (OOD) outliers prevalent in real-world applications. This discrepancy motivates Noisy TTA (NTTA), an online task to filter noisy OOD samples on the fly while maximizing in-distribution (ID) classification accuracy. Existing zero-shot NTTA approaches typically rely on test-time discriminative training, leading to overconfident misclassifications and significantly degraded inference efficiency. To address these limitations, we propose a novel framework named Dual Distribution Estimation (DDE), shifting the zero-shot NTTA paradigm from instance-level learning to training-free Gaussian distribution modeling. DDE incorporates two novel modules: Positive Feature Distribution Estimation (PFDE) and Negative Label Distribution Estimation (NLDE). PFDE explicitly models class-wise inclusion and exclusion Gaussian distributions to formulate a calibrated contrastive score, robustly enhancing ID accuracy. In parallel, NLDE improves OOD identification by explicitly modeling the negative label distribution to mine highly discriminative labels, effectively mitigating spurious correlations. Extensive experiments show that on the large-scale ImageNet benchmark, DDE achieves an improvement of 3.70\% in harmonic mean accuracy and reduces the FPR95 for OOD detection by 6.20\%, while ensuring highly scalable and efficient online inference. Furthermore, DDE is zero-shot and training-free, demonstrating remarkable robustness in data-scarce scenarios. Codes are available at https://github.com/ZhuWenjie98/DDE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
