vMFCoOp：在统一超球面流形上实现平衡的提示生物医学VLMs
vMFCoOp: Towards Equilibrium on a Unified Hyperspherical Manifold for Prompting Biomedical VLMs
AI总结：
vMFCoOp通过统一超球面流形上的vMF分布估计，实现LLM与CLIP主干间的语义对齐，提升生物医学VLMs的提示效果和少样本分类性能。
摘要：
最近基于大语言模型（LLM）的上下文优化（CoOp）方法利用医疗语义先验知识，为适应基于CLIP的生物医学视觉-语言模型（VLMs）提供了可扩展的替代方案，以替代手动提示工程和完全微调。然而，在这种背景下，提示学习受到LLM与CLIP变体之间语义错位的挑战，由于训练语料库和模型架构的差异；它进一步缺乏在持续演变的基础模型家族中的可扩展性。更关键的是，通过传统欧几里得空间优化的成对多模态对齐缺乏建模统一表示或应用局部几何约束的能力，这往往会加剧复杂生物医学成像中的模态差距，并破坏少样本适应。在本工作中，我们提出了vMFCoOp框架，该框架在共享的超球面流形上反向估计von Mises-Fisher（vMF）分布，通过统一语义锚点对任意LLM和CLIP主干之间的语义偏差进行对齐，以实现稳健的生物医学提示和优越的少样本分类。基于三个互补的约束，vMFCoOp在14个医学数据集、12种医学成像模态和13个解剖区域上表现出一致的改进，优于最先进的方法在准确性、泛化能力和临床适用性方面。这项工作旨在持续扩展以涵盖更多下游应用，相应的资源将通过https://github.com/VinyehShaw/UniEqui共享。
英文摘要：
Recent advances in context optimization (CoOp) guided by large language model (LLM)-distilled medical semantic priors offer a scalable alternative to manual prompt engineering and full fine-tuning for adapting biomedical CLIP-based vision-language models (VLMs). However, prompt learning in this context is challenged by semantic misalignment between LLMs and CLIP variants due to divergent training corpora and model architectures; it further lacks scalability across continuously evolving families of foundation models. More critically, pairwise multimodal alignment via conventional Euclidean-space optimization lacks the capacity to model unified representations or apply localized geometric constraints, which tends to amplify modality gaps in complex biomedical imaging and destabilize few-shot adaptation. In this work, we propose vMFCoOp, a framework that inversely estimates von Mises-Fisher (vMF) distributions on a shared Hyperspherical Manifold, aligning semantic biases between arbitrary LLMs and CLIP backbones via Unified Semantic Anchors to achieve robust biomedical prompting and superior few-shot classification. Grounded in three complementary constraints, vMFCoOp demonstrates consistent improvements across 14 medical datasets, 12 medical imaging modalities, and 13 anatomical regions, outperforming state-of-the-art methods in accuracy, generalization, and clinical applicability. This work aims to continuously expand to encompass more downstream applications, and the corresponding resources are intended to be shared through https://github.com/VinyehShaw/UniEqui.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
