BioMedVR: 面向生物医学视觉重编程的混淆感知提示专家混合模型
BioMedVR: Confusion-Aware Mixture-of-Prompt Experts for Biomedical Visual Reprogramming
AI总结：
提出BioMedVR框架，通过可学习的VR模块实现预训练VLM在生物医学图像上的少样本适应，利用混淆最小化机制和提示专家混合模型解决细粒度分类中的类间混淆问题，在18个数据集上验证了优越性能。
摘要：
最近视觉语言模型（如CLIP）的进展展示了其在自然图像领域的强大泛化能力。然而，将这些模型适应到生物医学成像并非易事：全模型微调计算成本高，而医学数据通常稀缺且表现出细微的细粒度类间差异，使得参数高效适应尤为关键。视觉重编程（VR）通过在输入空间中注入可学习扰动提供了一种参数高效的替代方案，但现有的针对VLM的VR方法主要关注正类提示，忽略了混淆负类，导致在细粒度医学场景中预测校准不佳。我们提出了BioMedVR，这是首个基于VR的生物医学成像框架，通过紧凑的可学习VR模块实现预训练VLM的少样本适应。为了缓解类混淆，我们引入了一种混淆最小化机制，利用LLM生成的混淆感知属性以及混淆抑制损失，明确减少假阳性对齐。此外，设计的提示专家混合模型结合了用于主类区分的正专家和用于混淆抑制的负专家，通过自适应门控进行平衡。在18个数据集（包括11个生物医学数据集和7个自然图像基准）上的大量实验表明，BioMedVR实现了优越的准确性和泛化能力，有效桥接了生物医学领域中的VR和VLM。
英文摘要：
Recent advances in vision-language models (VLMs) such as CLIP have demonstrated strong generalization across natural-image domains. However, adapting these models to biomedical imaging is non-trivial: full-model fine-tuning is computationally expensive, while medical data are often scarce and exhibit subtle, fine-grained inter-class differences, making parameter-efficient adaptation particularly critical. Visual Reprogramming (VR) offers a parameter-efficient alternative by injecting learnable perturbations into the input space, but existing VR approaches for VLMs mainly focus on positive class prompts and overlook confusing negatives, leading to miscalibrated predictions in fine-grained medical scenarios. We present BioMedVR, the first VR-based framework for biomedical imaging, enabling few-shot adaptation of pretrained VLMs through compact learnable VR modules. To mitigate class confusion, we introduce a Confusion Minimization Mechanism that leverages LLM-generated confusion-aware attributes together with a Confusion-Suppression Loss to explicitly reduce false-positive alignment. Moreover, the designed Mixture-of-Prompt Experts combines a positive expert for main-class discrimination and a negative expert for confusion suppression, balanced via adaptive gating. Extensive experiments on 18 datasets, including 11 biomedical datasets and 7 natural image benchmarks, demonstrate that BioMedVR achieves superior accuracy and generalization, effectively bridging VR and VLMs in biomedical domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
