发表机构
Tongji University（同济大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对初级护理中糖尿病视网膜病变筛查模型的部署问题，提出OrthKD框架，从异构教师中选择性蒸馏知识，在资源受限设备上实现实用筛查，提升模型性能，如MobileNetV3学生模型在相关数据集上取得更好的指标。
AI 中文摘要
在初级护理中部署糖尿病视网膜病变（DR）筛查模型需要在域转移下保持准确、安全和可靠的边缘高效系统。多教师知识蒸馏（KD）是一种自然的压缩策略，但现有方法大多假设所有教师提供同样可靠的监督。在我们的设置中，这一假设不成立。因此提出OrthKD，一种选择性信任蒸馏框架，它从强大的CNN转移全面监督，从弱ViT进行仅特征蒸馏，并强制教师特定的学生投影之间的正交性。在132,049张视网膜图像上，540万参数的MobileNetV3学生模型在EyePACS上达到0.885 QWK，将零样本Messidor-2性能从0.507提高到0.728 QWK，同时还实现了强大的转诊AUC和校准。这些结果表明，选择性地蒸馏异构教师可以在资源受限设备上实现实用的DR筛查。
英文摘要
Deploying diabetic retinopathy (DR) screening models in primary care requires edge-efficient systems that remain accurate, safe, and reliable under domain shift. Multi-teacher knowledge distillation (KD) is a natural compression strategy, but existing approaches largely assume that all teachers provide equally trustworthy supervision. In our setting, this assumption fails: a strong CNN teacher (EfficientNet-B3, 0.876 QWK) and a weaker Transformer teacher (Swin-Base, 0.830 QWK) are complementary, yet the Transformer's logits can still mislead the student. We therefore propose OrthKD, a selective-trust distillation framework that transfers full supervision from the strong CNN, uses feature-only distillation from the weak ViT, and enforces orthogonality between teacher-specific student projections to encourage complementary rather than redundant evidence. This design preserves local lesion precision, injects global structural context, and improves robustness to distribution shift. On 132,049 retinal images, a 5.4M-parameter MobileNetV3 student reaches 0.885 QWK on EyePACS and improves zero-shot Messidor-2 performance from 0.507 to 0.728 QWK, while also achieving strong referral AUC and calibration. These results show that selectively distilling heterogeneous teachers can enable practical DR screening on resource-constrained devices.
CommentsAccepted to the IJCAI-ECAI 2026 Special Track on AI and Health. 8 pages, 3 figures
