发表机构
Massachusetts Institute of Technology; Dana-Farber Cancer Institute; Brigham and Women’s Hospital; Harvard Medical School（麻省理工学院; 达纳-法伯癌症研究所; 布莱根妇女医院; 哈佛医学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出SAGE框架，可从ABMIL模型提取全局语义解释，在7个TCGA癌症队列的生存预测中恢复预后特征，揭示癌症特异性生物学，为病理学家解释模型行为提供支持。
AI 中文摘要
基于注意力的多实例学习（ABMIL）是计算病理学中用于玻片级预测的主流方法，但其注意力图仅能提供局部解释：它们指示模型关注的位置，却无法说明哪些组织学特征驱动了模型的预测，也无法说明模型在患者队列中的表现。我们提出语义注意力全局解释（SAGE），这是一种事后框架，可从冻结的ABMIL模型中提取基于语言的全局解释。利用病理学视觉语言模型，SAGE针对25个组织学概念的字典对图像块进行评分，根据模型学习到的注意力聚合这些评分，并量化每个概念与队列中预测风险的关联。将SAGE应用于7个TCGA癌症队列和3个基础模型的生存预测，它恢复了已确立的预后特征，如坏死的不良关联，同时揭示了癌症特异性生物学，包括肾细胞癌中与已知分子亚型一致的有利血管生成特征。消融研究表明，这些关联依赖于模型学习到的注意力，而非仅概念的普遍性，且概念字典捕获了基础模型特征编码的大部分预后信息。通过基于语义的解释，SAGE提供了一种可扩展、与模型无关的框架，用于理解ABMIL生存模型学习到的内容，使病理学家能够在队列层面解释模型行为，并为生物标志物识别提供潜力。
英文摘要
Attention-based multiple instance learning (ABMIL) is the predominant approach for slide-level prediction in computational pathology, yet its attention maps provide only local explanations: they indicate where a model focuses but not which histological features drive its predictions or how the model behaves across a patient cohort. We present Semantic Attention Global Explanations (SAGE), a post-hoc framework that extracts global, language-grounded explanations from a frozen ABMIL model. Using a pathology vision-language model, SAGE scores image patches against a dictionary of 25 histological concepts, aggregates these scores according to the model's learned attention, and quantifies how each concept relates to prediction risk across a cohort. Applied to survival prediction using seven TCGA cancer cohorts and three foundation models, SAGE recovered established prognostic features, such as the adverse association of necrosis, while revealing cancer-specific biology, including a favorable angiogenic signature in renal cell carcinoma consistent with known molecular subtypes. Ablation studies demonstrated that these associations depend on the model's learned attention rather than concept prevalence alone, and that the concept dictionary captures much of the prognostic information encoded by the foundation model features. Through semantically-grounded explanations, SAGE provides a scalable, model-agnostic framework for understanding what ABMIL survival models learn, enabling pathologists to interpret model behavior at the cohort level and offering the potential for biomarker identification.
CommentsProceedings of the MICCAI Workshop on Interpretability of Machine Intelligence in Medical Image Computing (iMIMIC)
