Deep learning has played a major role in the interpretation of dermoscopic images for detecting skin defects and abnormalities. However, current deep learning solutions for dermatological lesion analysis are typically limited in providing probabilistic predictions which highlights the importance of concerning uncertainties. This concept of uncertainty can provide a confidence level for each feature which prevents overconfident predictions with poor generalization on unseen data. In this paper, we propose an overall framework that jointly considers dermatological classification and uncertainty estimation together. The estimated confidence of each feature to avoid uncertain feature and undesirable shift, which are caused by environmental difference of input image, in the latent space is pooled from confidence network. Our qualitative results show that modeling uncertainties not only helps to quantify model confidence for each prediction but also helps classification layers to focus on confident features, therefore, improving the accuracy for dermatological lesion classification. We demonstrate the potential of the proposed approach in two state-of-the-art dermoscopic datasets (ISIC 2018 and ISIC 2019).
Duke University（杜克大学）；Duke Institute for Health Innovation（杜克健康创新研究所）
We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images, which can be indistinctly acquired via smartphone or dermoscopy capture. Additionally, we do not assume that images contain single lesions, thus the framework supports both focal or wide-field images. Specifically, we propose a two-stage approach in which we first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of…
Visual Bias and Interpretability in Deep Learning for Dermatological Image Analysis
Department of Computer Science and Engineering, European University of Bangladesh（计算机科学与工程系，孟加拉国欧洲大学）；Department of Computer Science and Engineering, BRAC University（计算机科学与工程系，BRAC大学）；Department of Information Systems, University of Maryland, Baltimore County（信息系统系，马里兰大学巴尔的摩县分校）
Skin lesion identification is a key step toward dermatological diagnosis. When describing a skin lesion, it is very important to note its body site distribution as many skin diseases commonly affect particular parts of the body. To exploit the correlation between skin lesions and their body site distributions, in this study, we investigate the possibility of improving skin lesion classification using the additional context information provided by body location. Specifically, we build a deep multi-task learning (MTL) framework to jointly optimize skin lesion classification and body location cla…
Ensemble Deep Learning and LLM-Assisted Reporting for Automated Skin Lesion Diagnosis
Institute of Computer Science The University of Agriculture Peshawar, Pakistan（计算机科学研究所 伊斯兰农业大学 薩瓦爾，巴基斯坦）；Department of Computer Science Islamia College Peshawar Pakistan（计算机科学系 伊斯兰学院 薩瓦爾，巴基斯坦）；Department of Software Engineering Islamia College Peshawar Pakistan（软件工程系 伊斯兰学院 薩瓦爾，巴基斯坦）
While deep learning based approaches have demonstrated expert-level performance in dermatological diagnosis tasks, they have also been shown to exhibit biases toward certain demographic attributes, particularly skin types (e.g., light versus dark), a fairness concern that must be addressed. We propose CIRCLe, a skin color invariant deep representation learning method for improving fairness in skin lesion classification. CIRCLe is trained to classify images by utilizing a regularization loss that encourages images with the same diagnosis but different skin types to have similar latent represent…
University of Bern（伯尔尼大学）；Inselspital, Bern University Hospital（伯尔尼大学医院 Inselspital）；Universitätsklinik Balgrist（巴尔格里斯特大学医院）
The detection of new or enlarged white-matter lesions in multiple sclerosis is a vital task in the monitoring of patients undergoing disease-modifying treatment for multiple sclerosis. However, the definition of 'new or enlarged' is not fixed, and it is known that lesion-counting is highly subjective, with high degree of inter- and intra-rater variability. Automated methods for lesion quantification hold the potential to make the detection of new and enlarged lesions consistent and repeatable. However, the majority of lesion segmentation algorithms are not evaluated for their ability to separa…
An analysis of data variation and bias in image-based dermatological datasets for machine learning classification
Centro de Informática, Universidade Federal de Pernambuco (UFPE)（伯南布哥联邦大学信息中心）；Ambulatório de Dermatologia, Hospital das Clinicas - UFPE（伯南布哥联邦大学临床医院皮肤科门诊）
Fudan University（复旦大学）；Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；Huashan Hospital, Fudan University（复旦大学附属华山医院）
Small lesions in magnetic resonance imaging (MRI) images are crucial for clinical diagnosis of many kinds of diseases. However, the MRI quality can be easily degraded by various noise, which can greatly affect the accuracy of diagnosis of small lesion. Although some methods for denoising MR images have been proposed, task-specific denoising methods for improving the diagnosis confidence of small lesions are lacking. In this work, we propose a voxel-wise hybrid residual MLP-CNN model to denoise three-dimensional (3D) MR images with small lesions. We combine basic deep learning architecture, MLP…
Whiting School of Engineering, Johns Hopkins University（约翰斯·霍普金斯大学怀廷工程学院）；School of Medicine, Johns Hopkins University（约翰斯·霍普金斯大学医学院）
Data-driven automatic approaches have demonstrated their great potential in resolving various clinical diagnostic dilemmas in neuro-oncology, especially with the help of standard anatomic and advanced molecular MR images. However, data quantity and quality remain a key determinant of, and a significant limit on, the potential of such applications. In our previous work, we explored synthesis of anatomic and molecular MR image network (SAMR) in patients with post-treatment malignant glioms. Now, we extend it and propose Confidence Guided SAMR (CG-SAMR) that synthesizes data from lesion informati…
Faculty of Informatics, University of Debrecen（德布勒森大学信息学院）
Skin cancer is a major public health problem, with over 5 million newly diagnosed cases in the United States each year. Melanoma is the deadliest form of skin cancer, responsible for over 9,000 deaths each year. In this paper, we propose an ensemble of deep convolutional neural networks to classify dermoscopy images into three classes. To achieve the highest classification accuracy, we fuse the outputs of the softmax layers of four different neural architectures. For aggregation, we consider the individual accuracies of the networks weighted by the confidence values provided by their final sof…
Skin cancer is considered to be the most common human malignancy. Around 5 million new cases of skin cancer are recorded in the United States annually. Early identification and evaluation of skin lesions is of great clinical significance, but the disproportionate dermatologist-patient ratio poses significant problem in most developing nations. Therefore a deep learning based architecture, known as SkiNet, is proposed with an objective to provide faster screening solution and assistance to newly trained physicians in the clinical diagnosis process. The main motive behind Skinet's design and dev…
Eastern Hepatobiliary Surgery Hospital（东方肝胆外科医院）；Ping An Technology（平安科技）；Georgia Institute of Technology（佐治亚理工学院）；PAII Inc.（PAII公司）
Objectives: to propose a fully-automatic computer-aided diagnosis (CAD) solution for liver lesion characterization, with uncertainty estimation. Methods: we enrolled 400 patients who had either liver resection or a biopsy and was diagnosed with either hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma, or secondary metastasis, from 2006 to 2019. Each patient was scanned with T1WI, T2WI, T1WI venous phase (T2WI-V), T1WI arterial phase (T1WI-A), and DWI MRI sequences. We propose a fully-automatic deep CAD pipeline that localizes lesions from 3D MRI studies using key-slice parsing an…
Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming
Computer Aided Medical Procedures (CAMP)（计算机辅助医疗程序）；TU Munich, Germany（慕尼黑工业大学，德国）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Munich, Germany（慕尼黑，德国）；Zhongshan Hospital, Fudan University, China（复旦大学中山医院）；The University of Hong Kong, Hongkong, China（香港大学，香港，中国）
Exploring the Challenge and Value of Deep Learning in Automated Skin Disease Diagnosis
Polytechnic Institute, Zhejiang University（浙江大学 polytechnic 院）；School of Social and Public Administration, East China University Of Science And Technology（东华大学社会科学与公共管理学院）；College of Science, National University of Defense Technology（国防科技大学科学学院）；School of Mathematical Sciences, Zhejiang University（浙江大学数学科学学院）
Centre for Computational Biology (CBIO), Mines Paris, PSL University（PSL大学巴黎矿业学院计算生物学中心（CBIO））；Tribun Health（Tribun健康公司）；Service de Pathologie, Hôpital Européen Georges-Pompidou, APHP（巴黎公共医疗集团乔治·蓬皮杜欧洲医院病理科）；Université Paris Cité（巴黎西岱大学）；Institut Curie, PSL University（PSL大学居里研究所）；INSERM（法国国家健康与医学研究院）
Grading precancerous lesions on whole slide images is a challenging task: the continuous space of morphological phenotypes makes clear-cut decisions between different grades often difficult, leading to low inter- and intra-rater agreements. More and more Artificial Intelligence (AI) algorithms are developed to help pathologists perform and standardize their diagnosis. However, those models can render their prediction without consideration of the ambiguity of the classes and can fail without notice which prevent their wider acceptance in a clinical context. In this paper, we propose a new score…
University of Electronic Science and Technology of China（电子科技大学）；University of Maryland School of Medicine（马里兰大学医学院）；Rutgers, the State University of New Jersey（罗格斯大学（新泽西州立大学））
Computed Tomography (CT) plays an important role in monitoring radiation-induced Pulmonary Fibrosis (PF), where accurate segmentation of the PF lesions is highly desired for diagnosis and treatment follow-up. However, the task is challenged by ambiguous boundary, irregular shape, various position and size of the lesions, as well as the difficulty in acquiring a large set of annotated volumetric images for training. To overcome these problems, we propose a novel convolutional neural network called PF-Net and incorporate it into a semi-supervised learning framework based on Iterative Confidence-…
University of Georgia（佐治亚大学）；Florida International University（佛罗里达国际大学）
Medical Imaging is one of the growing fields in the world of computer vision. In this study, we aim to address the Diabetic Retinopathy (DR) problem as one of the open challenges in medical imaging. In this research, we propose a new lesion detection architecture, comprising of two sub-modules, which is an optimal solution to detect and find not only the type of lesions caused by DR, their corresponding bounding boxes, and their masks; but also the severity level of the overall case. Aside from traditional accuracy, we also use two popular evaluation criteria to evaluate the outputs of our mod…
Robust Multicentre Detection and Classification of Colorectal Liver Metastases on CT: Application of Foundation Models
Maastricht University（马斯特里赫特大学）；Siemens Healthcare GmbH（西门子医疗有限公司）；The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院）；Medical University of Gdańsk（格但斯克医科大学）；Umeå University（于默奥大学）；Lithuanian University of Health Sciences（立陶宛健康科学大学）；University of Pisa（比萨大学）；Maastricht University Medical Center+（马斯特里赫特大学医学中心+）
Convolutional neural networks (CNNs) have achieved the state-of-the-art performance in skin lesion analysis. Compared with single CNN classifier, combining the results of multiple classifiers via fusion approaches shows to be more effective and robust. Since the skin lesion datasets are usually limited and statistically biased, while designing an effective fusion approach, it is important to consider not only the performance of each classifier on the training/validation dataset, but also the relative discriminative power (e.g., confidence) of each classifier regarding an individual sample in t…
GeoCFNet: Geometry-Aware Confidence Field Network for Robot-Assisted Endoscopic Submucosal Dissection
Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）；Theory Lab, Central Research Institute, 2012 Labs, Huawei Technologies Co. Ltd.（华为技术有限公司中央研究院2012实验室理论实验室）
ISTI-CNR（意大利国家研究委员会信息科学与技术研究所）；University of Pisa（比萨大学）；Criteo AI Lab（Criteo人工智能实验室）；Scuola Normale Superiore of Pisa（比萨高等师范学校）
A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we analyze a case study on skin lesion images where we customize an existing XAI approach for explaining a deep learning model able to recognize different types of skin lesions. The explanation is formed by…
Geisel School of Medicine at Dartmouth（达特茅斯盖泽尔医学院）；Dartmouth College（达特茅斯学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希区柯克医学中心）
In this paper, we demonstrate the feasibility and performance of deep residual neural networks for volumetric segmentation of irreversibly damaged brain tissue lesions on T1-weighted MRI scans for chronic stroke patients. A total of 239 T1-weighted MRI scans of chronic ischemic stroke patients from a public dataset were retrospectively analyzed by 3D deep convolutional segmentation models with residual learning, using a novel zoom-in&out strategy. Dice similarity coefficient (DSC), Average symmetric surface distance (ASSD), and Hausdorff distance (HD) of the identified lesions were measured by…
Deep neural networks have become the gold-standard approach for the automated segmentation of 3D medical images. Their full acceptance by clinicians remains however hampered by the lack of intelligible uncertainty assessment of the provided results. Most approaches to quantify their uncertainty, such as the popular Monte Carlo dropout, restrict to some measure of uncertainty in prediction at the voxel level. In addition not to be clearly related to genuine medical uncertainty, this is not clinically satisfying as most objects of interest (e.g. brain lesions) are made of groups of voxels whose …
National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）
Automated segmentation of tuberculosis (TB)-consistent lesions in chest X-rays (CXRs) using deep learning (DL) methods can help reduce radiologist effort, supplement clinical decision-making, and potentially result in improved patient treatment. The majority of works in the literature discuss training automatic segmentation models using coarse bounding box annotations. However, the granularity of the bounding box annotation could result in the inclusion of a considerable fraction of false positives and negatives at the pixel level that may adversely impact overall semantic segmentation perform…
Robust and Generalisable Segmentation of Subtle Epilepsy-causing Lesions: a Graph Convolutional Approach
Helmholtz Center Munich（亥姆霍兹慕尼黑中心）；LMU Munich（慕尼黑大学）；UCL Great Ormond Street Institute for Child Health（伦敦大学学院大奥蒙德街儿童健康研究所）；King’s College London（伦敦国王学院）；Harvard Medical School（哈佛医学院）；Massachusetts Institute of Technology（麻省理工学院）；University College London（伦敦大学学院）
Discriminative convolutional neural networks (CNNs), for which a voxel-wise conditional Multinoulli distribution is assumed, have performed well in many brain lesion segmentation tasks. For a trained discriminative CNN to be used in clinical practice, the patient's radiological features are inputted into the model, in which case a conditional distribution of segmentations is produced. Capturing the uncertainty of the predictions can be useful in deciding whether to abandon a model, or choose amongst competing models. In practice, however, we never know the ground truth segmentation, and theref…
Faculty of Biology and Medicine, University of Lausanne (UNIL)（日内瓦大学生物医学学院）；Radiology Department, Lausanne University Hospital (CHUV)（拉索恩大学医院放射科）；MedGIFT, Institute of Informatics, School of Management, HES--SO Valais--Wallis University of Applied Sciences and Arts Western Switzerland（应用科学与艺术西瓦利斯-瓦利斯大学信息学院）；CIBM Center for Biomedical Imaging（生物医学成像中心）；Department of Radiology and Medical Informatics, University of Geneva（日内瓦大学放射科与医学信息学系）；Translational Imaging in Neurology (ThINK) Basel, Department of Medicine and Biomedical Engineering, University Hospital Basel and University of Basel（神经学转化成像（ThINK）巴塞尔，医学与生物医学工程系，巴塞尔大学医院和巴塞尔大学）；Multiple Sclerosis Center, Department of Neurology, University Hospital Basel（多发性硬化中心，神经科，巴塞尔大学医院）；Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel（临床神经免疫学和神经科学巴塞尔研究中心（RC2NB），巴塞尔大学医院和巴塞尔大学）；Department of Neurology, Icahn School of Medicine at Mount Sinai（伊坎医学院 Mount Sinai 神经科）；Translational Neuroradiology Section, National Institute of Neurological Disorders and Stroke, National Institutes of Health（神经学转化放射学部门，国家神经疾病与中风研究所，国家卫生研究院）；Neuroinflammation Imaging Lab (NIL), Université catholique de Louvain（神经炎症成像实验室（NIL），卢瓦纳大学）；EPFL+ECAL Lab, École polytechnique fédérale de Lausanne (EPFL)（EPFL+ECAL 实验室，日内瓦联邦理工学院（EPFL））
Medical image segmentation annotations suffer from inter- and intra-observer variations even among experts due to intrinsic differences in human annotators and ambiguous boundaries. Leveraging a collection of annotators' opinions for an image is an interesting way of estimating a gold standard. Although training deep models in a supervised setting with a single annotation per image has been extensively studied, generalizing their training to work with datasets containing multiple annotations per image remains a fairly unexplored problem. In this paper, we propose an approach to handle annotato…
Using Multiple Dermoscopic Photographs of One Lesion Improves Melanoma Classification via Deep Learning: A Prognostic Diagnostic Accuracy Study
German Cancer Research Center (DKFZ)（德国癌症研究中心）；University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg（海德堡鲁普雷希特-卡尔大学曼海姆医学中心）；DKFZ Hector Cancer Institute at the University Medical Center Mannheim（曼海姆医学中心DKFZ赫克托癌症研究所）；University Hospital Carl Gustav Carus, Technische Universität Dresden（德累斯顿工业大学卡尔·古斯塔夫·卡鲁斯大学医院）；University Hospital (UKSH), Kiel（基尔大学医院（UKSH））
