BDC-Adapter：布朗距离协方差用于更好的视觉-语言推理
BDC-Adapter: Brownian Distance Covariance for Better Vision-Language Reasoning
AI总结：
针对现有视觉-语言微调方法仅捕捉线性关系的问题，提出BDC-Adapter，引入布朗距离协方差建模所有可能关系，整合原型相似性与多模态推理，显著提升分类性能。
摘要：
大规模预训练的视觉-语言模型（VLMs），如CLIP和ALIGN，引入了学习可迁移视觉表示的新范式。近期，研究人员对开发轻量级微调技术以将这些模型适应下游视觉任务产生了浓厚兴趣。我们认识到，当前最先进的微调方法（如Tip-Adapter）仅考虑查询图像特征与支持集少样本训练样本特征之间的协方差，这只能捕捉线性关系，并可能引发对独立性的错误感知。为解决此问题，在本工作中，我们创新性地将布朗距离协方差（BDC）引入视觉-语言推理领域。BDC度量能够建模所有可能的关系，为衡量特征依赖性提供了稳健的度量。基于此，我们提出了一种名为BDC-Adapter的新方法，该方法整合了BDC原型相似性推理和多模态推理网络预测来执行分类任务。我们广泛的实验结果表明，所提出的BDC-Adapter能够自由处理非线性关系并充分表征独立性，大幅超越了当前最先进的方法。
英文摘要：
Large-scale pre-trained Vision-Language Models (VLMs), such as CLIP and ALIGN, have introduced a new paradigm for learning transferable visual representations. Recently, there has been a surge of interest among researchers in developing lightweight fine-tuning techniques to adapt these models to downstream visual tasks. We recognize that current state-of-the-art fine-tuning methods, such as Tip-Adapter, simply consider the covariance between the query image feature and features of support few-shot training samples, which only captures linear relations and potentially instigates a deceptive perception of independence. To address this issue, in this work, we innovatively introduce Brownian Distance Covariance (BDC) to the field of vision-language reasoning. The BDC metric can model all possible relations, providing a robust metric for measuring feature dependence. Based on this, we present a novel method called BDC-Adapter, which integrates BDC prototype similarity reasoning and multi-modal reasoning network prediction to perform classification tasks. Our extensive experimental results show that the proposed BDC-Adapter can freely handle non-linear relations and fully characterize independence, outperforming the current state-of-the-art methods by large margins.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
