SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images
Nanyang Technological University（南洋理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Vancouver Coast Health Research Institute（温哥华海岸健康研究所）；BC Cancer（不列颠哥伦比亚癌症中心）
Yongwei Wang, Yuheng Wang, Tim K. Lee, Chunyan Miao, Z. Jane Wang
Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Over the last few years, computer-aided diagnosis has been rapidly developed and make great progress in healthcare and medical practices due to the advances in artificial intelligence. However, most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. In this case, knowledge distillation (KD) has been proven as an efficient tool to help improve the adaptability of lightweight models under limited resources, meanwhile keeping a high-level representation capability. To bridge the gap, this study specifically proposes a novel method, termed SSD-KD, that unifies diverse knowledge into a generic KD framework for skin diseases classification. Our method models an intra-instance relational feature representation and integrates it with existing KD research. A dual relational knowledge distillation architecture is self-supervisedly trained while the weighted softened outputs are also exploited to enable the student model to capture richer knowledge from the teacher model. To demonstrate the effectiveness of our method, we conduct experiments on ISIC 2019, a large-scale open-accessed benchmark of skin diseases dermoscopic images. Experiments show that our distilled lightweight model can achieve an accuracy as high as 85% for the classification tasks of 8 different skin diseases with minimal parameters and computing requirements. Ablation studies confirm the effectiveness of our intra- and inter-instance relational knowledge integration strategy. Compared with state-of-the-art knowledge distillation techniques, the proposed method demonstrates improved performances for multi-diseases classification on the large-scale dermoscopy database.
Hospital Clínic Barcelona, Universitat de Barcelona, IDIBAPS（巴塞罗那临床医院，巴塞罗那大学，IDIBAPS）；IBM Research AI, T J Watson Research Center（IBM研究AI，T J Watson研究中心）；Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center（医学部皮肤科，纪念斯隆-凯特琳癌症中心）；Kitware；Signal Theory and Communications, Universitat Politècnica de Catalunya（信号理论与通信，加泰罗尼亚理工大学）；Melanoma Unit, Dermatology Department, Hospital Clínic Barcelona, Universitat de Barcelona, IDIBAPS（黑素瘤科，皮肤科，巴塞罗那临床医院，巴塞罗那大学，IDIBAPS）
Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)
Louisiana State University in Shreveport（路易斯安那州立大学什里夫波特分校）；Kitware（Kitware公司）；Memorial Sloan-Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）；Missouri University of Science and Technology（密苏里科技大学）；Emory University（埃默里大学）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Mitigating the Influence of Domain Shift in Skin Lesion Classification: A Benchmark Study of Unsupervised Domain Adaptation Methods on Dermoscopic Images
German Aerospace Center (DLR)（德国航空航天中心）；German Cancer Research Center (DKFZ)（德国癌症研究中心）
GraphDerm: Fusing Imaging, Physical Scale, and Metadata in a Population-Graph Classifier for Dermoscopic Lesions
School of Computer Science, Institute for Research in Fundamental Sciences (IPM)（计算机科学学院，基础科学研究所（IPM））；Department of Physics, Shahid Beheshti University（物理系，肖德·贝赫谢什提大学）；Faculty of Science, University of Tehran（科学学院，德黑兰大学）；Fatemeh Zahra Hospital, Alborz University of Medical Sciences（法特梅·扎拉医院，阿尔bor兹医学科学大学）；Department of Electrical and Computer Engineering, Sungkyunkwan University（电气与计算机工程系，成均馆大学）；Department of Computer Science and Engineering, Shahid Beheshti University（计算机科学与工程系，肖德·贝赫谢什提大学）；Autoimmune Bullous Diseases Research Center, Razi Hospital, Tehran University of Medical Sciences（自身免疫性水疱病研究中心，拉齐医院，德黑兰医学科学大学）
Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data
Hamburg University of Technology（汉堡工业大学）；Forschungszentrum Medizintechnik Hamburg（汉堡医疗技术研究中心）；University Medical Center Hamburg-Eppendorf（汉堡-埃彭多夫大学医学中心）
Deep Clustering via Center-Oriented Margin Free-Triplet Loss for Skin Lesion Detection in Highly Imbalanced Datasets
Amasya University（阿马西亚大学）；Bilkent University（比尔肯特大学）；National Magnetic Resonance Research Center, Bilkent University（比尔肯特大学国家磁共振研究中心）；Sabuncu Brain Research Center, Bilkent University（比尔肯特大学萨本朱脑研究中心）
Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification
Victoria University（维多利亚大学）；Bangaldesh University of Business and Technology（孟加拉国商业与技术大学）；Daffodil Institute of Information Technology（达芙妮信息技术学院）；American International University-Bangladesh (AIUB)（孟加拉国美国国际大学）；Multimedia University（多媒体大学）
USL-Net: Uncertainty Self-Learning Network for Unsupervised Skin Lesion Segmentation
School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）；Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education（教育部可持续城市智能交通工程研究中心）；National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University（西南交通大学综合交通大数据应用技术国家工程实验室）；Manufacturing Industry Chains Collaboration and Information Support Technology Key Laboratory of Sichuan Province, Southwest Jiaotong University（西南交通大学四川省制造业产业链协同与信息化支撑技术重点实验室）；Data Recovery Key Laboratory of Sichuan Province, Neijiang Normal University（内江师范学院四川省数据恢复重点实验室）
Ensemble of Convolutional Neural Networks for Dermoscopic Images Classification
Spanish National Centre for Biotechnology (CNB–CSIC)（西班牙国家生物技术中心（CNB–CSIC））；University of Novi Sad（诺威萨大学）；NTNU（挪威科技大学）；Uppsala University（乌普萨拉大学）；Mathematical Institute of the Serbian Academy of Sciences and Arts（塞尔维亚科学与艺术学院数学研究所）
Investigating and Exploiting Image Resolution for Transfer Learning-based Skin Lesion Classification
Medical University of Vienna（维也纳医科大学）；Loughborough University（拉夫堡大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；TissueGnostics GmbH（TissueGnostics有限公司）
Early Melanoma Diagnosis with Sequential Dermoscopic Images
Department of Epidemiology and Biostatistics, College of Public Health, Zhengzhou University（郑州大学公共卫生学院流行病学与生物统计学系）；Victorian Melanoma Service, Alfred Health（阿尔弗雷德健康服务中心维多利亚黑色素瘤中心）；eResearch Centre, Monash University（莫纳什大学电子研究中心）
Dermoscopic Image Classification with Neural Style Transfer
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Novartis Pharmaceuticals（诺华制药）；University of California, Irvine（加州大学欧文分校）；Wolfram Research（沃尔夫勒姆研究公司）
Unsupervised Skin Lesion Segmentation via Structural Entropy Minimization on Multi-Scale Superpixel Graphs
Beihang University（北京航空航天大学）；Zhongguancun Laboratory（中关村实验室）；Salesforce AI Research（赛富时人工智能研究院）；Didi Chuxing（滴滴出行）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Lehigh University（理海大学）
Hybrid Ensemble of Segmentation-Assisted Classification and GBDT for Skin Cancer Detection with Engineered Metadata and Synthetic Lesions from ISIC 2024 Non-Dermoscopic 3D-TBP Images
A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Context
Memorial Sloan Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）；The University of Queensland（昆士兰大学）；University of Athens Medical School（雅典大学医学院）；IBM Research（IBM研究院）；Universitat de Barcelona（巴塞罗那大学）；IDIBAPS（IDIBAPS（生物医学研究所））；Melanoma Institute Australia（澳大利亚黑色素瘤研究所）；Emory University School of Medicine（埃默里大学医学院）；Medical University of Vienna（维也纳医科大学）；Mayo Clinic（梅奥诊所）；SUNY Downstate Medical School（纽约州立大学唐斯泰特医学院）；Stony Brook Medical School（石溪大学医学院）；Rabin Medical Center（拉宾医疗中心）；Weill Cornell Medical College（威尔康奈尔医学院）
