BayesAdapter：CLIP少样本适配中的增强不确定性估计
BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptation
AI总结：
针对现有CLIP适配器忽视不确定性估计质量的问题，提出基于贝叶斯推理的BayesAdapter方法，通过估计完整概率分布捕捉参数可变性，在校准和选择性分类等任务中实现了高质量的不确定性估计。
摘要：
大型预训练视觉语言模型（VLM）的出现代表了机器学习领域的范式转变，在广泛的视觉识别任务中取得了前所未有的成果。CLIP作为最受欢迎的VLM之一，在分类任务中展现出卓越的零样本和迁移学习能力。为将CLIP迁移到下游任务，适配器是一种参数高效的方法，与相关的提示学习方法不同，它无需通过大型模型进行反向传播。然而，现有CLIP适配器的开发以判别性能为目标，其不确定性估计的质量却被忽视了。本研究表明，最先进的CLIP适配器的判别性能并不总是与其不确定性估计能力相关，而不确定性估计能力对于实际场景中的安全部署至关重要。我们还证明，其中一种适配器是从更通用的概率框架中通过最大后验（MAP）推理得到的。基于这一观察，我们提出了BayesAdapter，它利用贝叶斯推理来估计完整的概率分布而非单个点，从而更好地捕捉参数空间中固有的可变性。在全面的实证评估中，我们的方法在预测中获得了高质量的不确定性估计，在校准和选择性分类方面表现突出。我们的代码将在论文被接收后公开。
英文摘要：
The emergence of large pre-trained vision-language models (VLMs) represents a paradigm shift in machine learning, with unprecedented results in a broad span of visual recognition tasks. CLIP, one of the most popular VLMs, has exhibited remarkable zero-shot and transfer learning capabilities in classification. To transfer CLIP to downstream tasks, adapters constitute a parameter-efficient approach that avoids backpropagation through the large model (unlike related prompt learning methods). However, CLIP adapters have been developed to target discriminative performance, and the quality of their uncertainty estimates has been overlooked. In this work we show that the discriminative performance of state-of-the-art CLIP adapters does not always correlate with their uncertainty estimation capabilities, which are essential for a safe deployment in real-world scenarios. We also demonstrate that one of such adapters is obtained through MAP inference from a more general probabilistic framework. Based on this observation we introduce BayesAdapter, which leverages Bayesian inference to estimate a full probability distribution instead of a single point, better capturing the variability inherent in the parameter space. In a comprehensive empirical evaluation we show that our approach obtains high quality uncertainty estimates in the predictions, standing out in calibration and selective classification. Our code will be publicly available upon acceptance of the paper.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
