ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology
University of Texas at Austin(德克萨斯大学奥斯汀分校);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Dell Children’s Medical Center(德尔儿童医疗中心);The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);University of Nevada, Reno(内华达大学里诺分校)
Healthcare models are transitioning from unimodal prediction toward multimodal reasoning over heterogeneous diagnostic inputs. In computational pathology, for complex tumor subtypes where morphology alone can be challenging to distinguish, pathology reports and molecular measurements may provide additional diagnostic evidence alongside whole-slide images, yet existing models often fail to clarify how diverse signals assemble into recognizable diagnostic concepts. We propose ConceptM$^3$oE (Concept Multimodal MoE), which embeds concept formation directly within interaction-aware mixture-of-experts (MoE) pathways. The architecture decomposes evidence into modality-specific, redundant, and synergistic experts, which are then projected into structured concept bottlenecks mapping latent features to a hierarchy of morphology and biomarker concepts. To prevent the information loss typical of interpretable bottlenecks, we utilize residual pathways within each expert to allow task-relevant signals to flow both through the concepts and directly to the final task prediction, so that high performance is maintained alongside interpretability. Across an institutional pediatric brain tumor cohort and a public glioma cohort, the framework delivers competitive performance to unconstrained models while producing reasoning traces validated by an independent neuropathologist. In data-limited regimes, ConceptM$^3$oE improves limited-data performance, increasing macro-F1 from 56.41% to 66.70% at small training sizes compared to non-concept-informed baselines, while also showing faster training convergence consistent with the regularizing effect of concept learning. This work offers a scalable path toward high-performance medical AI that is inherently verifiable and better aligned with the complex decision-making of clinical practice.
Pan-Cancer Integrative Histology-Genomic Analysis via Interpretable Multimodal Deep Learning
Brigham and Women’s Hospital(布莱根妇女医院);Harvard Medical School(哈佛医学院);Broad Institute of Harvard and MIT(哈佛-麻省理工学院博德研究所);Dana-Farber Cancer Institute(丹娜-法伯癌症研究所);Harvard University(哈佛大学)
PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMs
University of Science and Technology of China(中国科学技术大学);Shenzhen University(深圳大学);Nanyang Technological University(南洋理工大学);Imperial College London(伦敦帝国学院);Huazhong University of Science and Technology(华中科技大学)
Toward Auditable Neuro-Symbolic Reasoning in Pathology: SQL as an Explicit Trace of Evidence
Capital Normal University, Beijing, China(首都师范大学);Leibniz-Institut für Analytische Wissenschaften-ISAS, Dortmund, Germany(莱比锡分析科学研究所-ISAS);Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学深圳校区)
Evidence-based diagnostic reasoning with multi-agent copilot for human pathology
Health Sciences and Technology, Harvard-MIT(哈佛-MIT健康科学与技术);Department of Pathology, Massachusetts General Hospital, Harvard Medical School(麻省总医院病理科,哈佛医学院);Cancer Program, Broad Institute of Harvard and MIT(哈佛-MIT博德研究所癌症项目);Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保尔森工程与应用科学学院);Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT)(麻省理工学院电气工程与计算机科学);Harvard Data Science Initiative, Harvard University(哈佛大学数据科学计划)
Label-free Concept Based Multiple Instance Learning for Gigapixel Histopathology
University of Tübingen(蒂宾根大学);University of Lucerne(卢塞恩大学);Radboud University Medical Center(拉德堡德大学医学中心);Institut du Cancer de l’Ouest(西部癌症研究所);Oncode Institute(Oncode研究所)
Adaptive Diagnostic Reasoning Framework for Pathology with Multimodal Large Language Models
Computer Science Department, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校计算机科学系);Mechanical and Aerospace Engineering Department, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校机械与航空航天工程系);Department of Pathology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan(台湾国防医学院三军总医院病理部);Department of Pathology, National Taiwan University Hospital, Taipei, Taiwan(台湾国立台湾大学医院病理部);Department of Pathology, David Geffen School of Medicine, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校大卫·Geffen医学院病理部);Bioengineering Department, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校生物工程系);Institute for Quantitative and Computational Biosciences, University of California, CA, USA(加州大学定量与计算生物科学研究所)
SHAMSUL: Systematic Holistic Analysis to investigate Medical Significance Utilizing Local interpretability methods in deep learning for chest radiography pathology prediction
Stockholm University(斯德哥尔摩大学);Skatturinn (Iceland Revenue and Customs)(冰岛税务总局(冰岛税收和海关局))
The Next Layer: Augmenting Foundation Models with Structure-Preserving and Attention-Guided Learning for Local Patches to Global Context Awareness in Computational Pathology
The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);Northwestern University(西北大学)
Teaching AI Stepwise Diagnostic Reasoning with Report-Guided Chain-of-Thought Learning
Fujian University of Technology(福建工程学院);Fujian Provincial Key Laboratory of Big Data Mining and Applications(福建省大数据挖掘与应用重点实验室);Key Laboratory of Biomedical Imaging Science and System, Chinese Academy of Sciences(生物医学成像科学与系统重点实验室,中国科学院)
Multimodal Learning for Scalable Representation of High-Dimensional Medical Data
Kimia Lab, Dept. of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA(Kimia实验室,人工智能与信息学系,梅奥诊所,罗切斯特,MN,美国);Division of Computational Biology, Dept. of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA(计算生物学部门,定量健康科学系,梅奥诊所,罗切斯特,MN,美国)
CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation
Center for Artificial Intelligence in Medicine and Imaging(医学与影像人工智能中心);Department of Radiology(放射科);Department of Pediatrics(儿科);Department of Radiology, Weill Cornell Medicine(韦氏 Cornell 医学院放射科);Department of Radiology, Brigham and Women’s Hospital(布里奇妇产科医院放射科);University of California, Berkeley(加州大学伯克利分校);University of California, San Francisco(加州大学旧金山分校);Department of Biomedical Data Science(生物医学数据科学部)