Diagnostic Accuracy of Content Based Dermatoscopic Image Retrieval with Deep Classification Features
Philipp Tschandl, Giuseppe Argenziano, Majid Razmara, Jordan Yap
发表信息：
Tschandl P, Argenziano G, Razmara M, Yap J. Diagnostic Accuracy of Content Based Dermatoscopic Image Retrieval with Deep Classification Features. Br J Dermatol 2018 Sep 12. doi: 10.1111/bjd.17189
背景：通过神经网络对医学图像进行自动分类可以达到很高的准确率，但缺乏可解释性。目的：比较使用基于内容的图像检索（CBIR）检索视觉上相似的皮肤镜图像（带有相应的疾病标签）与神经网络预测的诊断准确性。研究方法：一个神经网络进行训练，预测疾病类别的皮肤镜图像从三个回顾性收集的图像数据集，分别包含888，2750和16691图像。诊断预测是基于视觉相似图像中最常见的诊断，或者基于网络输出的softmax的前1类预测。结果测量是预测恶性病变的ROC曲线下面积（AUC），多类准确度和平均精度（mAP），在相应数据集的未见过的测试图像上测量。结果如下：在所有三个数据集中，来自CBIR的皮肤癌预测（评估16个最相似的图像）显示出与softmax预测相似的AUC值（分别为0.842，0.806和0.852与0.830，0.810和0.847;所有p值>0.99）。同样，CBIR的多类准确性与softmax预测相当。当使用CBIR时，与softmax预测相比，训练用于检测仅3个类的网络在具有8个类的数据集上表现更好（mAP分别为0.184 vs. 0.368和0.198 vs. 0.403）。结论：基于神经网络的特征呈现视觉上相似的图像显示出与卷积神经网络的softmax基于概率的诊断相当的准确性。CBIR可能比softmax分类器更有助于提高临床医生在常规临床环境中的诊断准确性。
英文摘要：
Background: Automated classification of medical images through neural networks can reach high accuracy rates but lack interpretability.
Objectives: To compare the diagnostic accuracy obtained by using content based image retrieval (CBIR) to retrieve visually similar dermatoscopic images with corresponding disease labels against predictions made by a neural network.
Methods: A neural network was trained to predict disease classes on dermatoscopic images from three retrospectively collected image datasets containing 888, 2750 and 16691 images respectively. Diagnosis predictions were made based on the most commonly occurring diagnosis in visually similar images, or based on the top-1 class prediction of the softmax output from the network. Outcome measures were area under the ROC curve for predicting a malignant lesion (AUC), multiclass-accuracy and mean average precision (mAP), measured on unseen test images of the corresponding dataset.
Results: In all three datasets the skin cancer predictions from CBIR (evaluating the 16 most similar images) showed AUC values similar to softmax predictions (0.842, 0.806 and 0.852 versus 0.830, 0.810 and 0.847 respectively; p-value>0.99 for all). Similarly, the multiclass-accuracy of CBIR was comparable to softmax predictions. Networks trained for detecting only 3 classes performed better on a dataset with 8 classes when using CBIR as compared to softmax predictions (mAP 0.184 vs. 0.368 and 0.198 vs. 0.403 respectively).
Conclusions: Presenting visually similar images based on features from a neural network shows comparable accuracy to the softmax probability-based diagnoses of convolutional neural networks. CBIR may be more helpful than a softmax classifier in improving diagnostic accuracy of clinicians in a routine clinical setting.
Are We Recognizing the Jaguar or Its Background? A Diagnostic Framework for Jaguar Re-Identification
Hasso Plattner Institute（哈索·普拉特纳研究院）；NVIDIA（英伟达）；Jaguar ID Project（美洲豹识别项目）；Universidade Federal de Mato Grosso（马托格罗索联邦大学）；Technical University Munich（慕尼黑工业大学）；Helmholtz Zentrum Berlin（亥姆霍兹柏林研究中心）；Freie Universitaet Berlin（柏林自由大学）；Hyper3Labs
Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval
National Cancer Center Research Institute（国立癌症中心研究所）；RIKEN Center for Advanced Intelligent Project（理化学研究所先进智能项目中心）；The University of Tokyo（东京大学）；National Cancer Center Hospital（国立癌症中心医院）
Deep Metric Learning-based Image Retrieval System for Chest Radiograph and its Clinical Applications in COVID-19
Massachusetts General Hospital（麻省总医院）；Harvard University（哈佛大学）；MGH & BWH Center for Clinical Data Science（麻省总医院与布列根和妇女医院临床数据科学中心）；Kyungpook National University（庆北大学）；Yeungnam University College of Medicine（岭南大学医学院）；Catholic University of Daegu School of Medicine（大邱天主教大学医学院）；Keimyung University School of Medicine（启明大学医学院）
Content-Based Medical Image Retrieval with Opponent Class Adaptive Margin Loss
Amasya University（阿马西亚大学）；Bilkent University（比尔肯特大学）；National Magnetic Resonance Research Center, Bilkent University（比尔肯特大学国家磁共振研究中心）；Sabuncu Brain Research Center, Bilkent University（比尔肯特大学萨本库脑研究中心）
Histopathology WSI Encoding based on GCNs for Scalable and Efficient Retrieval of Diagnostically Relevant Regions
Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University（北京航空航天大学生物医学工程北京高级创新中心）；School of Software, Hefei University of Technology（合肥工业大学软件学院）；Tianjin Institute of Industrial Biotechnology, Chinese Academy of Sciences（中国科学院天津工业生物技术研究所）
Multimorbidity Content-Based Medical Image Retrieval Using Proxies
Monash University（莫纳什大学）；University of Melbourne（墨尔本大学）；Monash-Airdoc Research（莫纳什-鹰瞳研究中心）；Monash Medical AI（莫纳什医学人工智能中心）；Monash eResearch Centre（莫纳什电子研究中心）
Leveraging Medical Foundation Model Features in Graph Neural Network-Based Retrieval of Breast Histopathology Images
Artificial Intelligence Department, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran；Department of Computer Engineering Techniques, Mazaya University College, Nasiriyah, Iraq；Department of Medicine, Danube Private University, Krems an der Donau, Austria；Austrian Center for Medical Innovation；Research Center for Medical Image Analysis；Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, Austria
End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；Xinhua Hospital affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属新华医院）；Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属第九人民医院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）
Accurate and Scalable Multimodal Pathology Retrieval via Attentive Vision-Language Alignment
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程系）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属同济医院放射科）；Department of Pathology, Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属邵氏医院病理科）；Department of Pathology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属武汉中心医院病理科）；Department of Pathology, The First Affiliated Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属第一医院病理科）；Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学化学与生物工程系）；Division of Life Science, The Hong Kong University of Science and Technology（香港科学与技术大学生命科学系）
SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering
DAMO Academy, Alibaba Group（阿里巴巴达摩院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Hupan Lab（湖畔实验室）；School of Public Health, Zhejiang University（浙江大学公共卫生学院）
Retrieval-Guided Generation for Safer Histopathology Image Captioning
Kimia Lab, Department of Artificial Intelligence \& Informatics, Mayo Clinic, Rochester, MN, USA；Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, USA；Lane Department of Computer Science；Electrical Engineering, West Virginia University, Morgantown, WV, USA；Department of Computer Science；Engineering, Princeton University, Princeton, NJ, USA；Department of Computer Sciences, University of Wisconsin--Madison, Madison, WI, USA
