Libra-MIL：融合任务特定语言先验的多模态原型立体注入框架用于少样本全切片图像分类
Libra-MIL: Multimodal Prototypes Stereoscopic Infused with Task-specific Language Priors for Few-shot Whole Slide Image Classification
AI总结：
针对计算病理学中WSI少样本分类的模态协同与原型偏差问题，提出Libra-MIL方法，融合任务特定语言先验构建多模态原型并通过立体最优传输实现双向交互，提升模型泛化性与可解释性。
摘要：
尽管大语言模型（LLMs）在计算病理学中展现出广阔前景，但十亿像素级全切片图像（WSIs）的高昂计算成本要求使用多实例学习（MIL）以实现有效建模。一个关键挑战是病理任务通常仅提供包级标签，而LLMs生成的实例级描述因缺乏细粒度医学知识常存在偏差。为此，我们提出构建任务特定病理实体原型对学习可泛化特征和增强模型可解释性至关重要。此外，现有视觉-语言MIL方法常采用单向引导，限制了跨模态协同。本文提出一种新方法Libra-MIL，通过平衡信息压缩方案促进双向交互：利用冻结LLM生成任务特定病理实体描述并学习为文本原型，同时视觉分支学习实例级原型以减少模型对冗余数据的依赖；融合阶段采用基于相似性度量的立体最优传输（SOT）算法，在高维空间实现更广泛的语义对齐。我们在三个不同癌症数据集上进行少样本分类和可解释性实验，结果表明所提方法具有更优的泛化能力。
英文摘要：
While Large Language Models (LLMs) are emerging as a promising direction in computational pathology, the substantial computational cost of giga-pixel Whole Slide Images (WSIs) necessitates the use of Multi-Instance Learning (MIL) to enable effective modeling. A key challenge is that pathological tasks typically provide only bag-level labels, while instance-level descriptions generated by LLMs often suffer from bias due to a lack of fine-grained medical knowledge. To address this, we propose that constructing task-specific pathological entity prototypes is crucial for learning generalizable features and enhancing model interpretability. Furthermore, existing vision-language MIL methods often employ unidirectional guidance, limiting cross-modal synergy. In this paper, we introduce a novel approach, Multimodal Prototype-based Multi-Instance Learning, that promotes bidirectional interaction through a balanced information compression scheme. Specifically, we leverage a frozen LLM to generate task-specific pathological entity descriptions, which are learned as text prototypes. Concurrently, the vision branch learns instance-level prototypes to mitigate the model's reliance on redundant data. For the fusion stage, we employ the Stereoscopic Optimal Transport (SOT) algorithm, which is based on a similarity metric, thereby facilitating broader semantic alignment in a higher-dimensional space. We conduct few-shot classification and explainability experiments on three distinct cancer datasets, and the results demonstrate the superior generalization capabilities of our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
