发表机构
Sungkyunkwan University; Convergence Research Institute, Sungkyunkwan University（成均馆大学; 成均馆大学融合研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对非配对跨模态医学分类的挑战，提出SSCD方法，通过共享语义码本转移知识，在两个医学分类任务中提升学生模型性能，优于各基线方法
AI 中文摘要
跨模态知识蒸馏可将诊断知识从强大但成本高昂的教师模态转移到更廉价、更易部署的学生模态。然而在医学图像分析中，两种模态通常是非配对的：它们来自不同的患者队列，且占据几何上不兼容的特征空间，这使得实例级蒸馏无效，直接特征匹配也不可靠。为应对这些挑战，我们提出共享语义码本蒸馏（Shared Semantic Codebook Distillation, SSCD），通过共享离散码本比较教师与学生的表示。每张图像被表示为通用模态无关词汇上的分布，知识通过跨模态全局及类条件对齐这些分布进行转移，无需配对样本或直接可比的原始特征。码本通过指数移动平均线在线更新，通过熵正则化和死码重启保持多样性。推理时，所有教师侧和码本模块被丢弃，仅保留学生编码器和分类器。在两个异构非配对设置（OCT到眼底的视网膜疾病分类、CT到胸部X线的肺炎分类）中，SSCD分别将学生的宏F1从64.5提升至70.2、从73.8提升至76.3，在两种设置下均优于所有评估的蒸馏基线。代码和预训练模型可在this https URL获取
英文摘要
Cross-modal knowledge distillation can transfer diagnostic knowledge from a strong but costly teacher modality to a cheaper and more deployable student modality. In medical image analysis, however, the two modalities are often unpaired: they are collected from different patient cohorts and occupy geometrically incompatible feature spaces. This makes instance-level distillation invalid and direct feature matching unreliable. To address these challenges, we propose Shared Semantic Codebook Distillation (SSCD), which compares teacher and student representations through a shared discrete codebook. Each image is represented as a distribution over a common, modality-agnostic vocabulary, and knowledge is transferred by aligning these distributions across modalities, both globally and class-conditionally, without requiring paired samples or directly comparable raw features. The codebook is evolved online by exponential moving average and kept diverse through entropy regularization and dead-code restart. At inference, all teacher-side and codebook modules are discarded, leaving only the student encoder and classifier. On two heterogeneous unpaired settings, OCT-to-fundus retinal disease classification and CT-to-chest-X-ray pneumonia classification, SSCD improves the student from 64.5 to 70.2 macro-F1 and from 73.8 to 76.3 macro-F1, respectively, outperforming all evaluated distillation baselines on both settings. Code and pretrained models are available at https://github.com/DillanImans/SSCD-unpaired-distillation
Comments16 pages, 2 figures, 4 tables
