发表机构
School of Computer Science and Engineering, South China University of Technology; School of Computing and Mathematical Sciences, University of Leicester; Leicester Cancer Research Centre, University of Leicester; Department of Engineering Science, University of Oxford; Nuffield Department of Medicine, University of Oxford; Department of Computer Science and Engineering, The Hong Kong University of Science and Technology; ZoyMed（华南理工大学计算机科学与工程学院; 莱斯特大学计算与数学科学学院; 莱斯特大学莱斯特癌症研究中心; 牛津大学工程科学系; 牛津大学纳菲尔德医学院; 香港科技大学计算机科学与工程学系; 佐医医疗（ZoyMed））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出SmartStu框架，通过多教师集成蒸馏与对抗蒸馏等技术，定制出比通用PFMs小30倍以上且性能相当的乳腺癌专用病理学基础模型。
AI 中文摘要
病理学基础模型（Pathology Foundation Models, PFMs）能提供强大的组织表征，已成为数字病理学的核心。但在疾病专用场景中的部署受限于两点：一是拥有数十亿参数的PFMs计算成本高昂；二是从泛癌、多中心预训练中继承的分布不匹配与非生物偏差，包括位点特异性特征和疾病流行度不平衡，这些因素会促使捷径学习，且无法充分重视可靠建模特定癌症类型所需的细微形态特征。本文提出SmartStu（即“智能学生”），这是一种通过蒸馏定制紧凑的乳腺癌专用PFMs并缓解混杂因素的框架。SmartStu将多个教师PFMs的表征蒸馏到轻量学生骨干网络中，关键在于引入对抗蒸馏，利用专门的噪声模型（在蒸馏集上训练以预测以边缘为主的干扰线索），将该噪声模型作为反例，对抗目标鼓励学生识别并抑制可预测干扰目标的特征。我们还结合了多教师集成蒸馏和带人工制品注入的辅助自监督目标，在三个外部队列（Yale HER2、SLN-Breast和BRACS）上用多个小型骨干网络验证SmartStu。结果显示，SmartStu生成的乳腺癌专用PFMs比通用PFMs小30倍以上，同时在平衡准确率（balanced accuracy, bAcc）和AUC衡量的下游性能上基本保留甚至有所提升。代码可在this https URL获取。
英文摘要
Pathology foundation models (PFMs) provide strong tissue representations and have become central to digital pathology. However, deployment in disease-specific settings is limited by 1) the high computational cost of billion-parameter PFMs and 2) distribution mismatch and non-biological bias inherited from pan-cancer, multi-centre pre-training, including site-specific signatures and imbalanced disease prevalence. These factors can encourage shortcut learning and under-emphasise subtle morphology required for reliable modelling of a specific cancer type. We present SmartStu (a Smart Student), a framework to customise compact, breast-cancer-specific PFMs via distillation whilst mitigating confounding. SmartStu distils representations from multiple teacher PFMs into a lightweight student backbone. Crucially, we introduce adversarial distillation that leverages a dedicated noise model trained to predict nuisance, edge-dominated cues on the distillation set. Using this noise model as a counterexample, the adversarial objective encourages the student to recognise, yet suppress, features predictive of nuisance targets. We further incorporate multi-teacher ensemble distillation and an auxiliary self-supervised objective with artefact injection. We validate SmartStu on three external cohorts (Yale HER2, SLN-Breast, and BRACS) with multiple tiny backbones. SmartStu yields breast-cancer-specific PFMs that are over $30\times$ smaller than general PFMs whilst largely preserving, and sometimes improving, downstream performance measured by balanced accuracy (bAcc) and AUC. Code is available at https://github.com/zwchen03/advDistall.
Comments11 pages, 2 figures, 2 tables. Accepted to MICCAI 2026 (early accept)
