arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01547cs.CV

PathMoE:用于儿童脑肿瘤分类的可解释多模态交互专家

PathMoE: Interpretable Multimodal Interaction Experts for Pediatric Brain Tumor Classification

University of Texas(德克萨斯大学);Dell Children’s Medical Center(德尔儿童医学中心);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Nevada, Reno(内华达大学里诺分校)
Jian Yu, Joakim Nguyen, Jinrui Fang, Awais Naeem, Zeyuan Cao, Sanjay Krishnan, Nicholas Konz, Tianlong Chen, Chandra Krishnan, Hairong Wang, Edward Castillo, Yi… 展开作者

Jian Yu, Joakim Nguyen, Jinrui Fang, Awais Naeem, Zeyuan Cao, Sanjay Krishnan, Nicholas Konz, Tianlong Chen, Chandra Krishnan, Hairong Wang, Edward Castillo, Ying Ding, Ankita Shukla

AI总结:

PathMoE通过整合多模态信息提升儿童脑肿瘤分类性能,揭示了不同模态间的交互作用。

摘要:

准确分类儿童中枢神经系统肿瘤仍然具有挑战性,这归因于组织学复杂性和有限的训练数据。虽然病理基础模型已经推进了全切片图像(WSI)分析,但它们通常未能利用临床文本和组织微结构中丰富的互补信息。为此,我们提出了PathMoE,一种可解释的多模态框架,通过基于最新基础模型的交互感知混合专家架构,整合H&E切片、病理报告和细胞级细胞图。通过训练专门的专家来捕捉模态的独特性、冗余性和协同效应,PathMoE采用输入依赖的门控机制,动态加权这些交互,提供样本级可解释性。我们在内部儿童脑肿瘤数据集(PBT)和外部TCGA数据集上评估了我们的框架。在PBT上,整合WSI、文本和图模态时,PathMoE将宏F1从0.762提高到0.799(+0.037);在TCGA上,通过图知识增强WSI,宏F1从0.668提高到0.709(+0.041)。这些结果表明,相对于最先进的图像-only基线,性能有显著提升,同时揭示了驱动个体预测的具体模态交互。这种可解释性对于罕见肿瘤亚型尤其关键,其中透明的模型推理对于临床信任和诊断验证至关重要。

英文摘要:

Accurate classification of pediatric central nervous system tumors remains challenging due to histological complexity and limited training data. While pathology foundation models have advanced whole-slide image (WSI) analysis, they often fail to leverage the rich, complementary information found in clinical text and tissue microarchitecture. To this end, we propose PathMoE, an interpretable multimodal framework that integrates H\&E slides, pathology reports, and nuclei-level cell graphs via an interaction-aware mixture-of-experts architecture built on state-of-the-art foundation models for each modality. By training specialized experts to capture modality uniqueness, redundancy, and synergy, PathMoE employs an input-dependent gating mechanism that dynamically weights these interactions, providing sample-level interpretability. We evaluate our framework on two dataset-specific classification tasks on an internal pediatric brain tumor dataset (PBT) and external TCGA datasets. PathMoE improves macro-F1 from 0.762 to 0.799 (+0.037) on PBT when integrating WSI, text, and graph modalities; on TCGA, augmenting WSI with graph knowledge improves macro-F1 from 0.668 to 0.709 (+0.041). These results demonstrate significant performance gains over state-of-the-art image-only baselines while revealing the specific modality interactions driving individual predictions. This interpretability is particularly critical for rare tumor subtypes, where transparent model reasoning is essential for clinical trust and diagnostic validation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑