The Fast and Accurate Approach to Detection and Segmentation of Melanoma Skin Cancer using Fine-tuned Yolov3 and SegNet Based on Deep Transfer Learning
Melanoma is one of the most serious skin cancers that can occur in any part of the human skin. Early diagnosis of melanoma lesions will significantly increase their chances of being cured. Improving melanoma segmentation will help doctors or surgical robots remove the lesion more accurately from body parts. Recently, the learning-based segmentation methods achieved desired results in image segmentation compared to traditional algorithms. This study proposes a new approach to improve melanoma skin lesions detection and segmentation by defining a two-step pipeline based on deep learning models. Our methods were evaluated on ISIC 2018 (Skin Lesion Analysis Towards Melanoma Detection Challenge Dataset) well-known dataset. The proposed methods consist of two main parts for real-time detection of lesion location and segmentation. In the detection section, the location of the skin lesion is precisely detected by the fine-tuned You Only Look Once version 3 (F-YOLOv3) and then fed into the fine-tuned Segmentation Network (F-SegNet). Skin lesion localization helps to reduce the unnecessary calculation of whole images for segmentation. The results show that our proposed F-YOLOv3 performs better at 96% in mean Average Precision (mAP). Compared to state-of-the-art segmentation approaches, our F-SegNet achieves higher performance with 95.16% accuracy.
Melanoma is clinically difficult to distinguish from common benign skin lesions, particularly melanocytic naevus and seborrhoeic keratosis. The dermoscopic appearance of these lesions has huge intra-class variations and high inter-class visual similarities. Most current research is focusing on single-class segmentation irrespective of classes of skin lesions. In this work, we evaluate the performance of deep learning on multi-class segmentation of ISIC-2017 challenge dataset, which consists of 2,750 dermoscopic images. We propose an end-to-end solution using fully convolutional networks (FCNs)…
The prevalence of skin melanoma is rapidly increasing as well as the recorded death cases of its patients. Automatic image segmentation tools play an important role in providing standardized computer-assisted analysis for skin melanoma patients. Current state-of-the-art segmentation methods are based on fully convolutional neural networks, which utilize an encoder-decoder approach. However, these methods produce coarse segmentation masks due to the loss of location information during the encoding layers. Inspired by Pyramid Scene Parsing Network (PSP-Net), we propose an encoder-decoder model t…
University of Electronic Science and Technology of China（电子科技大学）
Dermoscopy image detection stays a tough task due to the weak distinguishable property of the object.Although the deep convolution neural network signifigantly boosted the performance on prevelance computer vision tasks in recent years,there remains a room to explore more robust and precise models to the problem of low contrast image segmentation.Towards the challenge of Lesion Segmentation in ISBI 2017,we built a symmetrical identity inception fully convolution network which is based on only 10 reversible inception blocks,every block composed of four convolution branches with combination of d…
University of Girona（赫罗纳大学）；Université Clermont Auvergne（克莱蒙奥弗涅大学）
Automatic segmentation of skin lesion is considered a crucial step in Computer Aided Diagnosis (CAD) for melanoma diagnosis. Despite its significance, skin lesion segmentation remains a challenging task due to their diverse color, texture, and indistinguishable boundaries and forms an open problem. Through this study, we present a new and automatic semantic segmentation network for robust skin lesion segmentation named Dermoscopic Skin Network (DSNet). In order to reduce the number of parameters to make the network lightweight, we used depth-wise separable convolution in lieu of standard convo…
Early detection of melanoma is difficult for the human eye but a crucial step towards reducing its death rate. Computerized detection of these melanoma and other skin lesions is necessary. The central research question in this paper is "How to segment skin lesion images using a neural network with low available data?". This question is divided into three sub questions regarding best performing network structure, training data and training method. First theory associated with these questions is discussed. Literature states that U-net CNN structures have excellent performances on the segmentatio…
Can We Go Beyond Visual Features? Neural Tissue Relation Modeling for Relational Graph Analysis in Non-Melanoma Skin Histology
Mohamed bin Zayed University of AI（Mohamed bin Zayed人工智能大学）；School of Information and Data Sciences（信息与数据科学学院）；Vellore Institute of Technology（维洛雷理工学院）；Loughborough University（洛桑大学）；MD Anderson Cancer Center, The University of Texas（MD安德森癌症中心，德克萨斯大学）
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Science and Technology of China（中国科学技术大学）
Skin lesion segmentation is an important step for automatic melanoma diagnosis. Due to the non-negligible diversity of lesions from different patients, extracting powerful context for fine-grained semantic segmentation is still challenging today. Although the deep convolutional neural network (CNNs) have made significant improvements on skin lesion segmentation, they often fail to reserve the spatial details and long-range dependencies context due to consecutive convolution striding and pooling operations inside CNNs. In this paper, we formulate a cascaded context enhancement neural network fo…
Convolutional Neural Networks have demonstrated human-level performance in the classification of melanoma and other skin lesions, but evident performance disparities between differing skin tones should be addressed before widespread deployment. In this work, we propose an efficient yet effective algorithm for automatically labelling the skin tone of lesion images, and use this to annotate the benchmark ISIC dataset. We subsequently use these automated labels as the target for two leading bias unlearning techniques towards mitigating skin tone bias. Our experimental results provide evidence tha…
Nanyang Technological University（南洋理工大学）；Central South University（中南大学）；Singapore University of Technology and Design（新加坡科技设计大学）
Deep learning techniques have shown their superior performance in dermatologist clinical inspection. Nevertheless, melanoma diagnosis is still a challenging task due to the difficulty of incorporating the useful dermatologist clinical knowledge into the learning process. In this paper, we propose a novel knowledge-aware deep framework that incorporates some clinical knowledge into collaborative learning of two important melanoma diagnosis tasks, i.e., skin lesion segmentation and melanoma recognition. Specifically, to exploit the knowledge of morphological expressions of the lesion region and …
Leveraging Pathology Foundation Models for Panoptic Segmentation of Melanoma in H&E Images
Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, United Kingdom（威斯敏斯特大学计算机科学系病理图像分析中心）；University Hospitals of Derby and Burton NHS Foundation Trust, United Kingdom（德比和伯顿国家健康服务基金会医院）；Department of Pathophysiology and Allergy Research, Center for Pathophysiology, Infectiology and Immunology, Medical University of Vienna, Austria（维也纳医学大学病理学与过敏研究系）；TissueGnostics GmbH, Vienna, Austria（维也纳TissueGnostics公司）
本文提出一种结合病理学基础模型Virchow2特征与Efficient-UNet的深度学习网络，用于黑色素瘤H&E图像五类组织分割，并在PUMA Grand Challenge中夺冠，验证了基础模型加速计算病理学流程的有效性。
Troy High School（特洛伊高中）；Harvard College（哈佛学院）；Beaumont Health（博蒙特健康）
As melanoma diagnoses increase across the US, automated efforts to identify malignant lesions become increasingly of interest to the research community. Segmentation of dermoscopic images is the first step in this process, thus accuracy is crucial. Although techniques utilizing convolutional neural networks have been used in the past for lesion segmentation, we present a solution employing the recently published DeepLab 3, an atrous convolution method for image segmentation. Although the results produced by this run are not ideal, with a mean Jaccard index of 0.498, we believe that with furthe…
Shahid Beheshti University（沙希德·贝赫什提大学）；RWTH Aachen University（亚琛工业大学）；Alzahra University（阿尔扎赫拉大学）；University of Regensburg（雷根斯堡大学）
Melanoma is caused by the abnormal growth of melanocytes in human skin. Like other cancers, this life-threatening skin cancer can be treated with early diagnosis. To support a diagnosis by automatic skin lesion segmentation, several Fully Convolutional Network (FCN) approaches, specifically the U-Net architecture, have been proposed. The U-Net model with a symmetrical architecture has exhibited superior performance in the segmentation task. However, the locality restriction of the convolutional operation incorporated in the U-Net architecture limits its performance in capturing long-range depe…
Over the last decades, the incidence of skin cancer, melanoma and non-melanoma, has increased at a continuous rate. In particular for melanoma, the deadliest type of skin cancer, early detection is important to increase patient prognosis. Recently, deep neural networks (DNNs) have become viable to deal with skin cancer detection. In this work, we present a smartphone-based application to assist on skin cancer detection. This application is based on a Convolutional Neural Network(CNN) trained on clinical images and patients demographics, both collected from smartphones. Also, as skin cancer dat…
Malignant melanoma is a common skin cancer that is mostly curable before metastasis -when growths spawn in organs away from the original site. Melanoma is the most dangerous type of skin cancer if left untreated due to the high risk of metastasis. This paper presents Melatect, a machine learning (ML) model embedded in an iOS app that identifies potential malignant melanoma. Melatect accurately classifies lesions as malignant or benign over 96.6% of the time with no apparent bias or overfitting. Using the Melatect app, users have the ability to take pictures of skin lesions (moles) and subseque…
Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）
Melanoma is a life-threatening form of skin cancer when left undiagnosed at the early stages. Although there are more cases of non-melanoma cancer than melanoma cancer, melanoma cancer is more deadly. Early detection of melanoma is crucial for the timely diagnosis of melanoma cancer and prohibit its spread to distant body parts. Segmentation of skin lesion is a crucial step in the classification of melanoma cancer from the cancerous lesions in dermoscopic images. Manual segmentation of dermoscopic skin images is very time consuming and error-prone resulting in an urgent need for an intelligent…
Kasetsart University Sriracha Campus（农业大学是拉差校区）；University of Hertfordshire（赫特福德大学）
This chapter presents a methodology for diagnosis of pigmented skin lesions using convolutional neural networks. The architecture is based on convolu-tional neural networks and it is evaluated using new CNN models as well as re-trained modification of pre-existing CNN models were used. The experi-mental results showed that CNN models pre-trained on big datasets for gen-eral purpose image classification when re-trained in order to identify skin le-sion types offer more accurate results when compared to convolutional neural network models trained explicitly from the dermatoscopic images. The bes…
Skin lesion datasets consist predominantly of normal samples with only a small percentage of abnormal ones, giving rise to the class imbalance problem. Also, skin lesion images are largely similar in overall appearance owing to the low inter-class variability. In this paper, we propose a two-stage framework for automatic classification of skin lesion images using adversarial training and transfer learning toward melanoma detection. In the first stage, we leverage the inter-class variation of the data distribution for the task of conditional image synthesis by learning the inter-class mapping a…
Australian e-Health Research Centre, CSIRO（澳大利亚联邦科学与工业研究组织电子健康研究中心）
Generative learning is a powerful tool for representation learning, and shows particular promise for problems in biomedical imaging. However, in this context, sampling from the distribution is secondary to finding representations of real images, which often come with labels and explicitly represent the content and quality of the target distribution. It remains difficult to faithfully reconstruct images from generative models, particularly those as complex as histological images. In this work, two existing methods (autoencoders and adversarial latent autoencoders) are combined in attempt to imp…
Application of Machine Learning in Melanoma Detection and the Identification of 'Ugly Duckling' and Suspicious Naevi: A Review
School of Electrical Engineering and Computer Science, University of Queensland（昆士兰大学电气工程与计算机科学学院）；Frazer Institute, The University of Queensland（昆士兰大学弗雷泽研究所）；Centre for Health Services Research, Faculty of Medicine, The University of Queensland（昆士兰大学医学院健康服务研究中心）
Melanoma is the most dangerous form of skin cancer, which is responsible for the majority of skin cancer-related deaths. Early diagnosis of melanoma can significantly reduce mortality rates and treatment costs. Therefore, skin cancer specialists are using image-based diagnostic tools for detecting melanoma earlier. We aim to develop a handheld device featured with low cost and high performance to enhance early detection of melanoma at the primary healthcare. But, developing this device is very challenging due to the complicated computations required by the embedded diagnosis system. Thus, we a…
Proscia Inc.（Proscia公司）；University of Florida College of Medicine（佛罗里达大学医学院）；Sidney Kimmel Medical College at Thomas Jefferson University（托马斯杰斐逊大学西德尼·金梅尔医学院）；Cedars-Sinai（西达赛奈医疗中心）
Although melanoma occurs more rarely than several other skin cancers, patients' long term survival rate is extremely low if the diagnosis is missed. Diagnosis is complicated by a high discordance rate among pathologists when distinguishing between melanoma and benign melanocytic lesions. A tool that allows pathology labs to sort and prioritize melanoma cases in their workflow could improve turnaround time by prioritizing challenging cases and routing them directly to the appropriate subspecialist. We present a pathology deep learning system (PDLS) that performs hierarchical classification of d…
Manchester Metropolitan University（曼彻斯特城市大学）；University of Auckland（奥克兰大学）
Early detection of skin cancer, particularly melanoma, is crucial to enable advanced treatment. Due to the rapid growth in the numbers of skin cancers, there is a growing need of computerized analysis for skin lesions. The state-of-the-art public available datasets for skin lesions are often accompanied with very limited amount of segmentation ground truth labeling as it is laborious and expensive. The lesion boundary segmentation is vital to locate the lesion accurately in dermoscopic images and lesion diagnosis of different skin lesion types. In this work, we propose the use of fully automat…
Melanoma is considered to be the most aggressive form of skin cancer. Due to the similar shape of malignant and benign cancerous lesions, doctors spend considerably more time when diagnosing these findings. At present, the evaluation of malignancy is performed primarily by invasive histological examination of the suspicious lesion. Developing an accurate classifier for early and efficient detection can minimize and monitor the harmful effects of skin cancer and increase patient survival rates. This paper proposes a multi-class classification task using the CoAtNet architecture, a hybrid model …
Among the different types of skin cancer, melanoma is considered to be the deadliest and is difficult to treat at advanced stages. Detection of melanoma at earlier stages can lead to reduced mortality rates. Desktop-based computer-aided systems have been developed to assist dermatologists with early diagnosis. However, there is significant interest in developing portable, at-home melanoma diagnostic systems which can assess the risk of cancerous skin lesions. Here, we present a smartphone application that combines image capture capabilities with preprocessing and segmentation to extract the As…
Institute for Molecular Bioscience, The University of Queensland（昆士兰大学分子生物科学研究所）；MyLab Pathology（迈莱博病理实验室）
Pathologists have a rich vocabulary with which they can describe all the nuances of cellular morphology. In their world, there is a natural pairing of images and words. Recent advances demonstrate that machine learning models can now be trained to learn high-quality image features and represent them as discrete units of information. This enables natural language, which is also discrete, to be jointly modelled alongside the imaging, resulting in a description of the contents of the imaging. Here we present experiments in applying discrete modelling techniques to the problem domain of non-melano…
Malignant melanoma is the deadliest form of skin cancer and, in recent years, is rapidly growing in terms of the incidence worldwide rate. The most effective approach to targeted treatment is early diagnosis. Deep learning algorithms, specifically convolutional neural networks, represent a methodology for the image analysis and representation. They optimize the features design task, essential for an automatic approach on different types of images, including medical. In this paper, we adopted pretrained deep convolutional neural networks architectures for the image representation with purpose t…
Automatic diagnosis of malignant melanoma highly depends on the segmentation methods used for the suspicious lesion. We suggest the parameter selection method (PSM) and maximum area method (MAM) for the segmentation of the lesion to be diagnosed. Herein, these segmentation methods are compared to a skin cancer expert's segmentation and three other conventional algorithms. The diagnosis of malignant melanoma based on the two suggested, three conventional, and expert's segmentation are compared with respect to sensitivity, specificity, and accuracy.
This paper provides the required description of the methods used to obtain submitted results for Task1 and Task 3 of ISIC 2018: Skin Lesion Analysis Towards Melanoma Detection. The results have been created by a team of researchers at the University of Dayton Signal and Image Processing Lab. In this submission, traditional classifiers with hand-crafted features are utilized for Task 1 and Task 3. Our team is providing additional separate submissions using deep learning methods for comparison.
Department of Epidemiology and Biostatistics, College of Public Health, Zhengzhou University（郑州大学公共卫生学院流行病学与生物统计学系）；Victorian Melanoma Service, Alfred Health（阿尔弗雷德健康服务中心维多利亚黑色素瘤中心）；eResearch Centre, Monash University（莫纳什大学电子研究中心）
Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transfo…
Skin lesions segmentation is an important step in the process of automated diagnosis of the skin melanoma. However, the accuracy of segmenting melanomas skin lesions is quite a challenging task due to less data for training, irregular shapes, unclear boundaries, and different skin colors. Our proposed approach helps in improving the accuracy of skin lesion segmentation. Firstly, we have introduced the coordinate convolutional layer before passing the input image into the encoder. This layer helps the network to decide on the features related to translation invariance which further improves the…
Priyadarshini Institute of Engineering & Technology（普里雅达希尼工程技术学院）
Skin cancer, a major form of cancer, is a critical public health problem with 123,000 newly diagnosed melanoma cases and between 2 and 3 million non-melanoma cases worldwide each year. The leading cause of skin cancer is high exposure of skin cells to UV radiation, which can damage the DNA inside skin cells leading to uncontrolled growth of skin cells. Skin cancer is primarily diagnosed visually employing clinical screening, a biopsy, dermoscopic analysis, and histopathological examination. It has been demonstrated that the dermoscopic analysis in the hands of inexperienced dermatologists may …
Xiamen University（厦门大学）；Xiamen University Malaysia（厦门大学马来西亚分校）；Manteia Technologies Co.,Ltd（曼泰亚科技有限公司）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）
Skin lesion segmentation from dermoscopy images is of great importance for improving the quantitative analysis of skin cancer. However, the automatic segmentation of melanoma is a very challenging task owing to the large variation of melanoma and ambiguous boundaries of lesion areas. While convolutional neutral networks (CNNs) have achieved remarkable progress in this task, most of existing solutions are still incapable of effectively capturing global dependencies to counteract the inductive bias caused by limited receptive fields. Recently, transformers have been proposed as a promising tool …
Amasya University（阿马西亚大学）；Bilkent University（比尔肯特大学）；National Magnetic Resonance Research Center, Bilkent University（比尔肯特大学国家磁共振研究中心）；Sabuncu Brain Research Center, Bilkent University（比尔肯特大学萨本朱脑研究中心）
Melanoma is a fatal skin cancer that is curable and has dramatically increasing survival rate when diagnosed at early stages. Learning-based methods hold significant promise for the detection of melanoma from dermoscopic images. However, since melanoma is a rare disease, existing databases of skin lesions predominantly contain highly imbalanced numbers of benign versus malignant samples. In turn, this imbalance introduces substantial bias in classification models due to the statistical dominance of the majority class. To address this issue, we introduce a deep clustering approach based on the …
K N Toosi University of Technology（K N Toosi工业大学）；University of Tehran（德黑兰大学）；Amirkabir University of Technology（Amirkabir工业大学）；Uniklinik RWTH Aachen（亚琛工业大学医院）
According to WHO[1], since the 1970s, diagnosis of melanoma skin cancer has been more frequent. However, if detected early, the 5-year survival rate for melanoma can increase to 99 percent. In this regard, skin lesion segmentation can be pivotal in monitoring and treatment planning. In this work, ten models and four augmentation configurations are trained on the ISIC 2016 dataset. The performance and overfitting are compared utilizing five metrics. Our results show that the U-Net-Resnet50 and the R2U-Net have the highest metrics value, along with two data augmentation scenarios. We also invest…
Support Vector Machine (SVM) is a common classifier used for efficient classification with high accuracy. SVM shows high accuracy for classifying melanoma (skin cancer) clinical images within computer-aided diagnosis systems used by skin cancer specialists to detect melanoma early and save lives. We aim to develop a medical low-cost handheld device that runs a real-time embedded SVM- based diagnosis system for use in primary care for early detection of melanoma. In this paper, an optimized SVM classifier is implemented onto a recent FPGA platform using the latest design methodology to be embed…
Memorial Sloan Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）；The University of Queensland（昆士兰大学）；University of Athens Medical School（雅典大学医学院）；IBM Research（IBM研究院）；Universitat de Barcelona（巴塞罗那大学）；IDIBAPS（IDIBAPS（生物医学研究所））；Melanoma Institute Australia（澳大利亚黑色素瘤研究所）；Emory University School of Medicine（埃默里大学医学院）；Medical University of Vienna（维也纳医科大学）；Mayo Clinic（梅奥诊所）；SUNY Downstate Medical School（纽约州立大学唐斯泰特医学院）；Stony Brook Medical School（石溪大学医学院）；Rabin Medical Center（拉宾医疗中心）；Weill Cornell Medical College（威尔康奈尔医学院）
Prior skin image datasets have not addressed patient-level information obtained from multiple skin lesions from the same patient. Though artificial intelligence classification algorithms have achieved expert-level performance in controlled studies examining single images, in practice dermatologists base their judgment holistically from multiple lesions on the same patient. The 2020 SIIM-ISIC Melanoma Classification challenge dataset described herein was constructed to address this discrepancy between prior challenges and clinical practice, providing for each image in the dataset an identifier …
Cleveland High School（克利夫兰高中）；University of Cambridge（剑桥大学）
Melanoma is not the most common form of skin cancer, but it is the most deadly. Currently, the disease is diagnosed by expert dermatologists, which is costly and requires timely access to medical treatment. Recent advances in deep learning have the potential to improve diagnostic performance, expedite urgent referrals and reduce burden on clinicians. Through smart phones, the technology could reach people who would not normally have access to such healthcare services, e.g. in remote parts of the world, due to financial constraints or in 2020, COVID-19 cancellations. To this end, we have invest…
Eötvös University（厄特沃什大学）；Wigner Research Centre for Physics（维格纳物理研究中心）；Semmelweis University（塞麦尔维斯大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
Our goal is to bridge human and machine intelligence in melanoma detection. We develop a classification system exploiting a combination of visual pre-processing, deep learning, and ensembling for providing explanations to experts and to minimize false negative rate while maintaining high accuracy in melanoma detection. Source images are first automatically segmented using a U-net CNN. The result of the segmentation is then used to extract image sub-areas and specific parameters relevant in human evaluation, namely center, border, and asymmetry measures. These data are then processed by tailore…
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)（德国人工智能研究中心）
We describe a software toolbox for the configuration of deep neural networks in the domain of skin cancer classification. The implemented software architecture allows developers to quickly set up new convolutional neural network (CNN) architectures and hyper-parameter configurations. At the same time, the user interface, manageable as a simple spreadsheet, allows non-technical users to explore different configuration settings that need to be explored when switching to different data sets. In future versions, meta leaning frameworks can be added, or AutoML systems that continuously improve over…
Melanoma, one of most dangerous types of skin cancer, re-sults in a very high mortality rate. Early detection and resection are two key points for a successful cure. Recent research has used artificial intelligence to classify melanoma and nevus and to compare the assessment of these algorithms to that of dermatologists. However, an imbalance of sensitivity and specificity measures affected the performance of existing models. This study proposes a method using deep convolutional neural networks aiming to detect melanoma as a binary classification problem. It involves 3 key features, namely cus…
SkinFormer: Learning Statistical Texture Representation with Transformer for Skin Lesion Segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shandong Computer Science Center（山东省计算中心）；Qilu University of Technology（齐鲁工业大学）；Shandong Fundamental Research Center for Computer Science（山东计算机科学基础研究中心）；Beijing University of Posts and Telecommunications（北京邮电大学）
This work is about the semantic segmentation of skin lesion boundary and their attributes using Image-to-Image Translation with Conditional Adversarial Nets. Melanoma is a type of skin cancer that can be cured if detected in time. Segmentation into dermoscopic images is an essential procedure for computer-assisted diagnosis due to its existing artifacts typical of skin images. To alleviate the image annotation process, we propose to use a modified Pix2Pix network. The discriminator network learns the mapping from a dermal image as an input and a mask image of six channels as an output. Likewis…
