We introduce SLIMP (Skin Lesion Image-Metadata Pre-training) for learning rich representations of skin lesions through a novel nested contrastive learning approach that captures complex relationships between images and metadata. Melanoma detection and skin lesion classification based solely on images, pose significant challenges due to large variations in imaging conditions (lighting, color, resolution, distance, etc.) and lack of clinical and phenotypical context. Clinicians typically follow a holistic approach for assessing the risk level of the patient and for deciding which lesions may be malignant and need to be excised, by considering the patient's medical history as well as the appearance of other lesions of the patient. Inspired by this, SLIMP combines the appearance and the metadata of individual skin lesions with patient-level metadata relating to their medical record and other clinically relevant information. By fully exploiting all available data modalities throughout the learning process, the proposed pre-training strategy improves performance compared to other pre-training strategies on downstream skin lesions classification tasks highlighting the learned representations quality.
Technische Universität Kaiserslautern（凯泽斯劳滕工业大学）；German Research Center for Artificial Intelligence GmbH (DFKI)（德国人工智能研究中心）
It is generally believed that the human visual system is biased towards the recognition of shapes rather than textures. This assumption has led to a growing body of work aiming to align deep models' decision-making processes with the fundamental properties of human vision. The reliance on shape features is primarily expected to improve the robustness of these models under covariate shift. In this paper, we revisit the significance of shape-biases for the classification of skin lesion images. Our analysis shows that different skin lesion datasets exhibit varying biases towards individual image …
In this paper, we studied extensively on different deep learning based methods to detect melanoma and skin lesion cancers. Melanoma, a form of malignant skin cancer is very threatening to health. Proper diagnosis of melanoma at an earlier stage is crucial for the success rate of complete cure. Dermoscopic images with Benign and malignant forms of skin cancer can be analyzed by computer vision system to streamline the process of skin cancer detection. In this study, we experimented with various neural networks which employ recent deep learning based models like PNASNet-5-Large, InceptionResNetV…
Convolutional neural networks (CNNs) have achieved great success in skin lesion classification. A balanced dataset is required to train a good model. However, due to the appearance of different skin lesions in practice, severe or even deadliest skin lesion types (e.g., melanoma) naturally have quite small amount represented in a dataset. In that, classification performance degradation occurs widely, it is significantly important to have CNNs that work well on class imbalanced skin lesion image dataset. In this paper, we propose SuperCon, a two-stage training strategy to overcome the class imba…
We present a method for skin lesion segmentation for the ISIC 2017 Skin Lesion Segmentation Challenge. Our approach is based on a Fully Convolutional Network architecture which is trained end to end, from scratch, on a limited dataset. Our semantic segmentation architecture utilizes several recent innovations in particularly in the combined use of (i) use of atrous convolutions to increase the effective field of view of the network's receptive field without increasing the number of parameters, (ii) the use of network-in-network $1\times1$ convolution layers to add capacity to the network and (…
Recent years have witnessed a rapid development of automated methods for skin lesion diagnosis and classification. Due to an increasing deployment of such systems in clinics, it has become important to develop a more robust system towards various Out-of-Distribution(OOD) samples (unknown skin lesions and conditions). However, the current deep learning models trained for skin lesion classification tend to classify these OOD samples incorrectly into one of their learned skin lesion categories. To address this issue, we propose a simple yet strategic approach that improves the OOD detection perfo…
Medical Image Analysis Lab, School of Computing Science（计算科学学院医学图像分析实验室）；Simon Fraser University（西蒙弗雷泽大学）
We present an automated approach to detect and longitudinally track skin lesions on 3D total-body skin surface scans. The acquired 3D mesh of the subject is unwrapped to a 2D texture image, where a trained objected detection model, Faster R-CNN, localizes the lesions within the 2D domain. These detected skin lesions are mapped back to the 3D surface of the subject and, for subjects imaged multiple times, we construct a graph-based matching procedure to longitudinally track lesions that considers the anatomical correspondences among pairs of meshes and the geodesic proximity of corresponding le…
ISTI-CNR（意大利国家研究委员会信息科学与技术研究所）；University of Pisa（比萨大学）；Criteo AI Lab（Criteo人工智能实验室）；Scuola Normale Superiore of Pisa（比萨高等师范学校）
A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we analyze a case study on skin lesion images where we customize an existing XAI approach for explaining a deep learning model able to recognize different types of skin lesions. The explanation is formed by…
International Institute of Information Technology Naya Raipur（国际信息技术学院纳亚赖普尔分校）
Skin cancer is one of the most prevalent forms of human cancer. It is recognized mainly visually, beginning with clinical screening and continuing with the dermoscopic examination, histological assessment, and specimen collection. Deep convolutional neural networks (CNNs) perform highly segregated and potentially universal tasks against a classified finegrained object. This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative Adversarial Networks) are utilized to tackle the class imbalance. T…
Exploring the Impact of Skin Color on Skin Lesion Segmentation
School of Digital and Physical Sciences at the University of Hull（赫尔大学数字与物理科学学院）；Faculty of Electrical Engineering at the University of Sarajevo（萨拉热窝大学电气工程学院）
This paper proposes an innovative method for segmentation of skin lesions in dermoscopy images developed by the authors, based on fuzzy classification of pixels and histogram thresholding.
kin cancer is considered one of the most common type of cancer in several countries. Due to the difficulty and subjectivity in the clinical diagnosis of skin lesions, Computer-Aided Diagnosis systems are being developed for assist experts to perform more reliable diagnosis. The clinical analysis and diagnosis of skin lesions relies not only on the visual information but also on the context information provided by the patient. This work addresses the problem of pigmented skin lesions detection from smartphones captured images. In addition to the features extracted from images, patient context i…
This paper summarizes our method and validation results for the ISIC Challenge 2018 - Skin Lesion Analysis Towards Melanoma Detection - Task 1: Lesion Segmentation
Skin lesions segmentation is an important step in the process of automated diagnosis of the skin melanoma. However, the accuracy of segmenting melanomas skin lesions is quite a challenging task due to less data for training, irregular shapes, unclear boundaries, and different skin colors. Our proposed approach helps in improving the accuracy of skin lesion segmentation. Firstly, we have introduced the coordinate convolutional layer before passing the input image into the encoder. This layer helps the network to decide on the features related to translation invariance which further improves the…
Skin cancer is by far the most common type of cancer. Early detection is the key to increase the chances for successful treatment significantly. Currently, Deep Neural Networks are the state-of-the-art results on automated skin cancer classification. To push the results further, we need to address the lack of annotated data, which is expensive and require much effort from specialists. To bypass this problem, we propose using Generative Adversarial Networks for generating realistic synthetic skin lesion images. To the best of our knowledge, our results are the first to show visually-appealing s…
This paper summarizes our method and validation results for the ISBI Challenge 2017 - Skin Lesion Analysis Towards Melanoma Detection - Part I: Lesion Segmentation
We present a deep learning approach to the ISIC 2017 Skin Lesion Classification Challenge using a multi-scale convolutional neural network. Our approach utilizes an Inception-v3 network pre-trained on the ImageNet dataset, which is fine-tuned for skin lesion classification using two different scales of input images.
This article presents the design, experiments and results of our solution submitted to the 2018 ISIC challenge: Skin Lesion Analysis Towards Melanoma Detection. We design a pipeline using state-of-the-art Convolutional Neural Network (CNN) models for a Lesion Boundary Segmentation task and a Lesion Diagnosis task.
Our system addresses Part 1, Lesion Segmentation and Part 3, Lesion Classification of the ISIC 2017 challenge. Both algorithms make use of deep convolutional networks to achieve the challenge objective.
Isfahan University of Technology（伊斯法罕理工大学）；University of British Columbia（不列颠哥伦比亚大学）；University of Michigan（密歇根大学）；McMaster University（麦克马斯特大学）
One of the essential tasks in medical image analysis is segmentation and accurate detection of borders. Lesion segmentation in skin images is an essential step in the computerized detection of skin cancer. However, many of the state-of-the-art segmentation methods have deficiencies in their border detection phase. In this paper, a new class of fully convolutional network is proposed, with new dense pooling layers for segmentation of lesion regions in skin images. This network leads to highly accurate segmentation of lesions on skin lesion datasets which outperforms state-of-the-art algorithms …
Microsoft（微软）；Cupertino High School（库比蒂诺高中）；Inglemoor High School（英格尔穆尔高中）
All datasets contain some biases, often unintentional, due to how they were acquired and annotated. These biases distort machine-learning models' performance, creating spurious correlations that the models can unfairly exploit, or, contrarily destroying clear correlations that the models could learn. With the popularity of deep learning models, automated skin lesion analysis is starting to play an essential role in the early detection of Melanoma. The ISIC Archive is one of the most used skin lesion sources to benchmark deep learning-based tools. Bissoto et al. experimented with different boun…
University of Brasília（巴西利亚大学）；Faculdade de Engenharias do Gama（伽马工程学院）
Skin lesions are conditions that appear on a patient due to many different reasons. One of these can be because of an abnormal growth in skin tissue, defined as cancer. This disease plagues more than 14.1 million patients and had been the cause of more than 8.2 million deaths, worldwide. Therefore, the construction of a classification model for 12 lesions, including Malignant Melanoma and Basal Cell Carcinoma, is proposed. Furthermore, in this work, it is used a ResNet-152 architecture, which was trained over 3,797 images, later augmented by a factor of 29 times, using positional, scale, and l…
Skin cancer diagnosis using NIR spectroscopy data of skin lesions in vivo using machine learning algorithms
Federal University of Espírito Santo (UFES)（圣埃斯皮里图联邦大学）；Secretary of Health of the Espírito Santo State（圣埃斯皮里图州卫生部）；University Hospital Cassiano Antônio Moraes (HUCAM)（卡西亚诺·安东尼奥·莫赖斯大学医院）
In this paper we approach the problem of skin lesion segmentation using a convolutional neural network based on the U-Net architecture. We present a set of training strategies that had a significant impact on the performance of this model. We evaluated this method on the ISIC Challenge 2018 - Skin Lesion Analysis Towards Melanoma Detection, obtaining threshold Jaccard index of 77.5%.
Skin lesion identification is a key step toward dermatological diagnosis. When describing a skin lesion, it is very important to note its body site distribution as many skin diseases commonly affect particular parts of the body. To exploit the correlation between skin lesions and their body site distributions, in this study, we investigate the possibility of improving skin lesion classification using the additional context information provided by body location. Specifically, we build a deep multi-task learning (MTL) framework to jointly optimize skin lesion classification and body location cla…
ISTI-CNR（意大利国家研究委员会信息科学与技术研究所）；University of Pisa（比萨大学）；Sorbonne Universite（索邦大学）
Explainable AI consists in developing mechanisms allowing for an interaction between decision systems and humans by making the decisions of the formers understandable. This is particularly important in sensitive contexts like in the medical domain. We propose a use case study, for skin lesion diagnosis, illustrating how it is possible to provide the practitioner with explanations on the decisions of a state of the art deep neural network classifier trained to characterize skin lesions from examples. Our framework consists of a trained classifier onto which an explanation module operates. The l…
Shanghai Jiao Tong University（上海交通大学）；École de technologie supérieure, Université du Québec（魁北克大学高等技术学院）；University of Leicester（莱斯特大学）
Skin lesion detection in dermoscopic images is essential in the accurate and early diagnosis of skin cancer by a computerized apparatus. Current skin lesion segmentation approaches show poor performance in challenging circumstances such as indistinct lesion boundaries, low contrast between the lesion and the surrounding area, or heterogeneous background that causes over/under segmentation of the skin lesion. To accurately recognize the lesion from the neighboring regions, we propose a dilated scale-wise feature fusion network based on convolution factorization. Our network is designed to simul…
University of Burgundy（勃艮第大学）；University of Cassino（卡西诺大学）；University of Girona（赫罗纳大学）
Skin lesion segmentation is one of the first steps towards automatic Computer-Aided Diagnosis of skin cancer. Vast variety in the appearance of the skin lesion makes this task very challenging. The contribution of this paper is to apply a power foreground extraction technique called GrabCut for automatic skin lesion segmentation with minimal human interaction in HSV color space. Preprocessing was performed for removing the outer black border. Jaccard Index was measured to evaluate the performance of the segmentation method. On average, 0.71 Jaccard Index was achieved on 1000 images from ISIC c…
The segmentation of skin lesions is a crucial task in clinical decision support systems for the computer aided diagnosis of skin lesions. Although deep learning-based approaches have improved segmentation performance, these models are often susceptible to class imbalance in the data, particularly, the fraction of the image occupied by the background healthy skin. Despite variations of the popular Dice loss function being proposed to tackle the class imbalance problem, the Dice loss formulation does not penalize misclassifications of the background pixels. We propose a novel metric-based loss f…
In this study, we investigate what a practically useful approach is in order to achieve robust skin disease diagnosis. A direct approach is to target the ground truth diagnosis labels, while an alternative approach instead focuses on determining skin lesion characteristics that are more visually consistent and discernible. We argue that, for computer-aided skin disease diagnosis, it is both more realistic and more useful that lesion type tags should be considered as the target of an automated diagnosis system such that the system can first achieve a high accuracy in describing skin lesions, an…
There has been a steady increase in the incidence of skin cancer worldwide, with a high rate of mortality. Early detection and segmentation of skin lesions are crucial for timely diagnosis and treatment, necessary to improve the survival rate of patients. However, skin lesion segmentation is a challenging task due to the low contrast of lesions and their high similarity in terms of appearance, to healthy tissue. This underlines the need for an accurate and automatic approach for skin lesion segmentation. To tackle this issue, we propose a convolutional neural network (CNN) called SkinNet. The …
In this work, we explore the issue of the inter-annotator agreement for training and evaluating automated segmentation of skin lesions. We explore what different degrees of agreement represent, and how they affect different use cases for segmentation. We also evaluate how conditioning the ground truths using different (but very simple) algorithms may help to enhance agreement and may be appropriate for some use cases. The segmentation of skin lesions is a cornerstone task for automated skin lesion analysis, useful both as an end-result to locate/detect the lesions and as an ancillary task for …
This paper summarizes our method and validation results for part 1 of the ISBI Challenge 2018. Our algorithm makes use of deep encoder-decoder network and novel skin lesion data augmentation to segment the challenge objective. Besides, we also propose an effective testing strategy by applying multi-model comparison.
This paper describes the participation of Araguaia Medical Vision Lab at the International Skin Imaging Collaboration 2017 Skin Lesion Challenge. We describe the use of deep convolutional neural networks in attempt to classify images of Melanoma and Seborrheic Keratosis lesions. With use of finetuned GoogleNet and AlexNet we attained results of 0.950 and 0.846 AUC on Seborrheic Keratosis and Melanoma respectively.
VIT University（VIT大学）；SSN College of Engineering（SSN工程学院）
Skin cancer is the most common of all cancers and each year million cases of skin cancer are treated. Treating and curing skin cancer is easy, if it is diagnosed and treated at an early stage. In this work we propose an automatic technique for skin lesion segmentation in dermoscopic images which helps in classifying the skin cancer types. The proposed method comprises of two major phases (1) preprocessing and (2) segmentation using semi-supervised learning algorithm. In the preprocessing phase noise are removed using filtering technique and in the segmentation phase skin lesions are segmented …
Convolutional Neural Networks have demonstrated human-level performance in the classification of melanoma and other skin lesions, but evident performance disparities between differing skin tones should be addressed before widespread deployment. In this work, we propose an efficient yet effective algorithm for automatically labelling the skin tone of lesion images, and use this to annotate the benchmark ISIC dataset. We subsequently use these automated labels as the target for two leading bias unlearning techniques towards mitigating skin tone bias. Our experimental results provide evidence tha…
This report summarises our method and validation results for the ISIC Challenge 2018 - Skin Lesion Analysis Towards Melanoma Detection - Task 1: Lesion Segmentation. We present a two-stage method for lesion segmentation with optimised training method and ensemble post-process. Our method achieves state-of-the-art performance on lesion segmentation and we win the first place in ISIC 2018 task1.
University of Campinas（坎皮纳斯大学）；Instituto Superior Técnico（里斯本高等理工学院）
Deep Learning failure cases are abundant, particularly in the medical area. Recent studies in out-of-distribution generalization have advanced considerably on well-controlled synthetic datasets, but they do not represent medical imaging contexts. We propose a pipeline that relies on artifacts annotation to enable generalization evaluation and debiasing for the challenging skin lesion analysis context. First, we partition the data into levels of increasingly higher biased training and test sets for better generalization assessment. Then, we create environments based on skin lesion artifacts to …
Convolutional neural networks (CNNs) have achieved the state-of-the-art performance in skin lesion analysis. Compared with single CNN classifier, combining the results of multiple classifiers via fusion approaches shows to be more effective and robust. Since the skin lesion datasets are usually limited and statistically biased, while designing an effective fusion approach, it is important to consider not only the performance of each classifier on the training/validation dataset, but also the relative discriminative power (e.g., confidence) of each classifier regarding an individual sample in t…
A Multi-Task Deep Learning Framework for Skin Lesion Classification, ABCDE Feature Quantification, and Evolution Simulation
Department of Chemistry, University of Cambridge, Cambridge, United Kingdom（剑桥大学化学系，剑桥，英国）；Harvard Ophthalmology AI Lab, Harvard Medical School, Boston, MA, USA（哈佛医学院眼科人工智能实验室，哈佛医学院，波士顿，马萨诸塞州，美国）
We proposed a two stage framework with only one network to analyze skin lesion images, we firstly trained a convolutional network to classify these images, and cropped the import regions which the network has the maximum activation value. In the second stage, we retrained this CNN with the image regions extracted from stage one and output the final probabilities. The two stage framework achieved a mean AUC of 0.857 in ISIC-2017 skin lesion validation set and is 0.04 higher than that of the original inputs, 0.821.
Applied Physics Laboratory, Johns Hopkins University（约翰斯·霍普金斯大学应用物理实验室）；Malone Center for Engineering in Healthcare, Johns Hopkins University（约翰斯·霍普金斯大学医疗保健工程马龙中心）
We examine progress in the use of AI for detecting skin lesions, with particular emphasis on the erythema migrans rash of acute Lyme disease, and other lesions, such as those from conditions like herpes zoster (shingles), tinea corporis, erythema multiforme, cellulitis, insect bites, or tick bites. We discuss important challenges for these applications, in particular the problems of AI bias regarding the lack of skin images in dark skinned individuals, being able to accurately detect, delineate, and segment lesions or regions of interest compared to normal skin in images, and low shot learning…
FH Joanneum University of Applied Sciences（约阿内姆应用科学大学）
Skin cancer is the most common cancer worldwide, with melanoma being the deadliest form. Dermoscopy is a skin imaging modality that has shown an improvement in the diagnosis of skin cancer compared to visual examination without support. We evaluate the current state of the art in the classification of dermoscopic images based on the ISIC-2019 Challenge for the classification of skin lesions and current literature. Various deep neural network architectures pre-trained on the ImageNet data set are adapted to a combined training data set comprised of publicly available dermoscopic and clinical im…
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Recent advances in automated skin cancer diagnosis have yielded performance on par with board-certified dermatologists. However, these approaches formulated skin cancer diagnosis as a simple classification task, dismissing the potential benefit from lesion segmentation. We argue that an accurate lesion segmentation can supplement the classification task with additive lesion information, such as asymmetry, border, intensity, and physical size; in turn, a faithful lesion classification can support the segmentation task with discriminant lesion features. To this end, this paper proposes a new mul…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Northwestern Polytechnical University（西北工业大学）
We recognize that the skin lesion diagnosis is an essential and challenging sub-task in Image classification, in which the Fisher vector (FV) encoding algorithm and deep convolutional neural network (DCNN) are two of the most successful techniques. Since the joint use of FV and DCNN has demonstrated proven success, the joint techniques could have discriminatory power on skin lesion diagnosis as well. To this hypothesis, we propose the aggregation algorithm for skin lesion diagnosis that utilize the residual network to extract the local features and the Fisher vector method to aggregate the loc…
For several skin conditions such as vitiligo, accurate segmentation of lesions from skin images is the primary measure of disease progression and severity. Existing methods for vitiligo lesion segmentation require manual intervention. Unfortunately, manual segmentation is time and labor-intensive, as well as irreproducible between physicians. We introduce a convolutional neural network (CNN) that quickly and robustly performs vitiligo skin lesion segmentation. Our CNN has a U-Net architecture with a modified contracting path. We use the CNN to generate an initial segmentation of the lesion, th…
CNR-ISASI（意大利国家研究委员会应用科学与智能系统研究所）；University of Salento（萨兰托大学）
This article presents a Deep CNN, based on the DenseNet architecture jointly with a highly discriminating learning methodology, in order to classify seven kinds of skin lesions: Melanoma, Melanocytic nevus, Basal cell carcinoma, Actinic keratosis / Bowen's disease, Benign keratosis, Dermatofibroma, Vascular lesion. In particular a 61 layers DenseNet, pre-trained on IMAGENET dataset, has been fine-tuned on ISIC 2018 Task 3 Challenge Dataset exploiting a Center Loss function.
McGill University（麦吉尔大学）；Universite de Montrea（蒙特利尔大学）；ETS Montreal（蒙特利尔高等工程学院）
In this paper, we study the problem of out-of-distribution detection in skin disease images. Publicly available medical datasets normally have a limited number of lesion classes (e.g. HAM10000 has 8 lesion classes). However, there exists a few thousands of clinically identified diseases. Hence, it is important if lesions not in the training data can be differentiated. Toward this goal, we propose DeepIF, a non-parametric Isolation Forest based approach combined with deep convolutional networks. We conduct comprehensive experiments to compare our DeepIF with three baseline models. Results demon…
Nanyang Technological University（南洋理工大学）；Central South University（中南大学）；Singapore University of Technology and Design（新加坡科技设计大学）
Deep learning techniques have shown their superior performance in dermatologist clinical inspection. Nevertheless, melanoma diagnosis is still a challenging task due to the difficulty of incorporating the useful dermatologist clinical knowledge into the learning process. In this paper, we propose a novel knowledge-aware deep framework that incorporates some clinical knowledge into collaborative learning of two important melanoma diagnosis tasks, i.e., skin lesion segmentation and melanoma recognition. Specifically, to exploit the knowledge of morphological expressions of the lesion region and …
Accurate segmentation of skin lesion from dermoscopic images is a crucial part of computer-aided diagnosis of melanoma. It is challenging due to the fact that dermoscopic images from different patients have non-negligible lesion variation, which causes difficulties in anatomical structure learning and consistent skin lesion delineation. In this paper, we propose a novel bi-directional dermoscopic feature learning (biDFL) framework to model the complex correlation between skin lesions and their informative context. By controlling feature information passing through two complementary directions,…
