发表机构
Concordia University; Mila – Quebec AI Institute（康考迪亚大学; 米拉-魁北克人工智能研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过分解分析和十个病理队列实验，揭示跨分辨率蒸馏中重构教师表示与下游性能存在差距，挑战重构误差作为转移度量，并提出诊断框架。
AI 中文摘要
跨分辨率知识蒸馏旨在通过转移高倍率表示来改进低倍率全切片分析，然而有用转移的条件仍不明确。我们开发了一种基于分解的分析方法，涵盖教师访问、表示损失和模型冗余，从而引出三个问题：(a) 教师目标是否有助于任务，(b) 低倍率学生能否预测这些目标，以及(c) 切片模型是否受益于这些预测。我们通过跨越十个病理队列的受控实验来研究这些问题，这些队列涵盖分类、分级和生存预测。在主要比较中，将教师区域均值与原生低倍率特征一起提供，在全部十个队列中均改善了下游性能。直接预测的重构误差低于残差预测，但预测的特征对教师目标中的变异表示不足。此外，更好的重构并不一致地改善下游分数，并且在预测的教师特征保持不变时，保留原生特征也会改变性能。总之，这些发现揭示了重构教师表示与其下游价值实现之间的差距。它们挑战了将重构误差作为跨分辨率转移充分度量的观点，并提供了一个诊断框架来检查转移在何处失效。未来的蒸馏设计必须同时考虑学生能预测什么以及切片模型如何使用这些预测。
英文摘要
Cross-resolution knowledge distillation aims to improve low-magnification whole- slide analysis by transferring high-magnification representations, yet the conditions for useful transfer remain unclear. We develop a decomposition-based analysis of teacher access, representation loss, and model excess, motivating three questions: whether (a) teacher targets help the task, (b) low-magnification students can predict them, and (c) slide models benefit from those predictions. We investigate them through controlled experiments across ten pathology cohorts spanning classifi- cation, grading, and survival prediction. In the main comparison, providing teacher regional means alongside native low-magnification features improves downstream performance in all ten cohorts. Direct prediction achieves lower reconstruction error than residual prediction, yet the predicted features underrepresent variation in the teacher targets. Moreover, better reconstruction does not consistently improve downstream scores, and retaining native features changes performance even when the predicted teacher features are held fixed. Together, these findings expose a gap between reconstructing teacher representations and realizing their downstream value. They challenge the sufficiency of reconstruction error as a measure of cross-resolution transfer and provide a diagnostic framework for examining where that transfer breaks down. Future distillation designs must account for both what students can predict and how slide models use those predictions.
CommentsUnder review as a conference paper at ICLR 2027
