CNNs, Transformers, Hybrid, and Vision Language Models for Skin Cancer Detection
Department of Computer Science and Software Engineering, Concordia University, Montreal, Canada（计算机科学与软件工程系，康科迪亚大学，加拿大蒙特利尔）；Ebovir Biotechnologie Inc., Montreal, Canada（Ebovir生物技术公司，加拿大蒙特利尔）
Durjoy Dey, Yuhong Yan, Hassan Hajjdiab
备注：
13 pages, 3 figures, accepted at ICPRAI 2026, The Fifth International Conference on Pattern Recognition and Artificial Intelligence. To appear in Lecture Notes in Computer Science
Skin cancer is a common and fast rising malignancy worldwide. Early detection is critical for improving outcomes. Deep learning models trained on dermoscopic and clinical images can support automated and fast triage. However, many studies evaluate only a limited set of architectures. Experimental setups also vary across studies. In this paper, we present a unified evaluation of twelve deep learning models for binary skin cancer detection on the PAD-UFES-20 dataset. The models span four families: convolutional neural networks (CNN), vision transformers (ViT), hybrid convolution transformer backbones, and vision language models (VLM). Performance is assessed using AUC, the maximum F1 score with its precision and recall, and sensitivity at 80% specificity, reflecting screening oriented requirements. Our results show that well tuned CNNs already provide strong baselines, but transformer based families consistently improve discrimination. Hybrid models (MaxViT Tiny, CoAtNet0) and a SigLIP based VLM achieve the best overall trade off between ranking performance and clinically relevant operating points, while CLIP based model offers high precision. The full codebase for all experiments is publicly released. Together, these findings offer practical guidance on which model families are most suitable for real world deployment in skin cancer screening and establish a reproducible reference point for future work on PAD-UFES-20.
Kennesaw State University（肯尼索州立大学）；University of Calabria（卡拉布里亚大学）
Skin cancer detection is challenging since different types of skin lesions share high similarities. This paper proposes a computer-based deep learning approach that will accurately identify different kinds of skin lesions. Deep learning approaches can detect skin cancer very accurately since the models learn each pixel of an image. Sometimes humans can get confused by the similarities of the skin lesions, which we can minimize by involving the machine. However, not all deep learning approaches can give better predictions. Some deep learning models have limitations, leading the model to a false…
As one kind of skin cancer, melanoma is very dangerous. Dermoscopy based early detection and recarbonization strategy is critical for melanoma therapy. However, well-trained dermatologists dominant the diagnostic accuracy. In order to solve this problem, many effort focus on developing automatic image analysis systems. Here we report a novel strategy based on deep learning technique, and achieve very high skin lesion segmentation and melanoma diagnosis accuracy: 1) we build a segmentation neural network (skin_segnn), which achieved very high lesion boundary detection accuracy; 2) We build anot…
VIT University（VIT大学）；SSN College of Engineering（SSN工程学院）
Skin cancer is the most common of all cancers and each year million cases of skin cancer are treated. Treating and curing skin cancer is easy, if it is diagnosed and treated at an early stage. In this work we propose an automatic technique for skin lesion segmentation in dermoscopic images which helps in classifying the skin cancer types. The proposed method comprises of two major phases (1) preprocessing and (2) segmentation using semi-supervised learning algorithm. In the preprocessing phase noise are removed using filtering technique and in the segmentation phase skin lesions are segmented …
This study focuses on automatic skin cancer detection using a Meta-learning approach for dermoscopic images. The aim of this study is to explore the benefits of the generalization of the knowledge extracted from non-medical data in the classification performance of medical data and the impact of the distribution shift problem within limited data by using a simple class and distribution balancer algorithm. In this study, a small sample of a combined dataset from 3 different sources was used to fine-tune a ResNet model pre-trained on non-medical data. The results show an increase in performance …
Using Computer Vision for Skin Disease Diagnosis in Bangladesh Enhancing Interpretability and Transparency in Deep Learning Models for Skin Cancer Classification
Skin cancer is the most common malignancy in the world. Automated skin cancer detection would significantly improve early detection rates and prevent deaths. To help with this aim, a number of datasets have been released which can be used to train Deep Learning systems - these have produced impressive results for classification. However, this only works for the classes they are trained on whilst they are incapable of identifying skin lesions from previously unseen classes, making them unconducive for clinical use. We could look to massively increase the datasets by including all possible skin …
Pakistan Navy Engineering College (PNEC)（巴基斯坦海军工程学院）；National University of Sciences & Technology (NUST)（国家科技大学）；Electronics and Power Engineering (EPE)（电子与动力工程系）
Technology aided platforms provide reliable tools in almost every field these days. These tools being supported by computational power are significant for applications that need sensitive and precise data analysis. One such important application in the medical field is Automatic Lesion Detection System (ALDS) for skin cancer classification. Computer aided diagnosis helps physicians and dermatologists to obtain a second opinion for proper analysis and treatment of skin cancer. Precise segmentation of the cancerous mole along with surrounding area is essential for proper analysis and diagnosis. …
Over the last decades, the incidence of skin cancer, melanoma and non-melanoma, has increased at a continuous rate. In particular for melanoma, the deadliest type of skin cancer, early detection is important to increase patient prognosis. Recently, deep neural networks (DNNs) have become viable to deal with skin cancer detection. In this work, we present a smartphone-based application to assist on skin cancer detection. This application is based on a Convolutional Neural Network(CNN) trained on clinical images and patients demographics, both collected from smartphones. Also, as skin cancer dat…
Dense object detection and temporal tracking are needed across applications domains ranging from people-tracking to analysis of satellite imagery over time. The detection and tracking of malignant skin cancers and benign moles poses a particularly challenging problem due to the general uniformity of large skin patches, the fact that skin lesions vary little in their appearance, and the relatively small amount of data available. Here we introduce a novel data synthesis technique that merges images of individual skin lesions with full-body images and heavily augments them to generate significant…
University of Waterloo（滑铁卢大学）；DarwinAI Corp.（DarwinAI公司）
Skin cancer continues to be the most frequently diagnosed form of cancer in the U.S., with not only significant effects on health and well-being but also significant economic costs associated with treatment. A crucial step to the treatment and management of skin cancer is effective skin cancer detection due to strong prognosis when treated at an early stage, with one of the key screening approaches being dermoscopy examination. Motivated by the advances of deep learning and inspired by the open source initiatives in the research community, in this study we introduce CancerNet-SCa, a suite of d…
GDN: A Stacking Network Used for Skin Cancer Diagnosis
School of Artificial Intelligence and automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；Biosystem Engineering, Zhejiang University（浙江大学生物系统工程学院）；Computer Science Department, New York University（纽约大学计算机科学系）；School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感信息工程学院）
Background: Skin cancer is one of the widely seen cancer worldwide and automatic classification of skin cancer can be benefited dermatology clinics for an accurate diagnosis. Hence, a machine learning-based automatic skin cancer detection model must be developed. Material and Method: This research interests to overcome automatic skin cancer detection problem. A colored skin cancer image dataset is used. This dataset contains 3297 images with two classes. An automatic multilevel textural and deep features-based model is presented. Multilevel fuse feature generation using discrete wavelet transf…
While skin cancer is the most diagnosed form of cancer in men and women, with more cases diagnosed each year than all other cancers combined, sufficiently early diagnosis results in very good prognosis and as such makes early detection crucial. While radiomics have shown considerable promise as a powerful diagnostic tool for significantly improving oncological diagnostic accuracy and efficiency, current radiomics-driven methods have largely rely on pre-defined, hand-crafted quantitative features, which can greatly limit the ability to fully characterize unique cancer phenotype that distinguish…
Skin cancer is the most common type of cancer. Specifically, melanoma is the cause of 75% of skin cancer deaths, although it is the least common skin cancer. Better detection of melanoma could have a positive impact on millions of people. The ISIC archive contains the largest publicly available collection of dermatoscopic images of skin lesions. In this research, we investigate the efficacy of applying advanced deep learning techniques in computer vision to identify melanoma in images of skin lesions. Through reviewing previous methods, including pre-trained models, deep-learning classifiers, …
Priyadarshini Institute of Engineering & Technology（普里雅达希尼工程技术学院）
Skin cancer, a major form of cancer, is a critical public health problem with 123,000 newly diagnosed melanoma cases and between 2 and 3 million non-melanoma cases worldwide each year. The leading cause of skin cancer is high exposure of skin cells to UV radiation, which can damage the DNA inside skin cells leading to uncontrolled growth of skin cells. Skin cancer is primarily diagnosed visually employing clinical screening, a biopsy, dermoscopic analysis, and histopathological examination. It has been demonstrated that the dermoscopic analysis in the hands of inexperienced dermatologists may …
Institute of Technology, Nirma University（尼尔玛大学技术学院）
Cancer, in general, is very deadly. Timely treatment of any cancer is the key to saving a life. Skin cancer is no exception. There have been thousands of Skin Cancer cases registered per year all over the world. There have been 123,000 deadly melanoma cases detected in a single year. This huge number is proven to be a cause of a high amount of UV rays present in the sunlight due to the degradation of the Ozone layer. If not detected at an early stage, skin cancer can lead to the death of the patient. Unavailability of proper resources such as expert dermatologists, state of the art testing fac…
The Institute of Cancer Research（英国癌症研究院）；Royal Free London（伦敦皇家自由医院）
The spectacular response observed in clinical trials of immunotherapy in patients with previously uncurable Melanoma, a highly aggressive form of skin cancer, calls for a better understanding of the cancer-immune interface. Computational pathology provides a unique opportunity to spatially dissect such interface on digitised pathological slides. Accurate cellular classification is a key to ensure meaningful results, but is often challenging even with state-of-art machine learning and deep learning methods. We propose a hierarchical framework, which mirrors the way pathologists perceive tumour …
Dartmouth College（达特茅斯学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希区柯克医疗中心）；Geisel School of Medicine（盖塞尔医学院）；Metrohealth System（都会医疗系统）；Case Western Reserve University（凯斯西储大学）
Recently, there has been great interest in developing Artificial Intelligence (AI) enabled computer-aided diagnostics solutions for the diagnosis of skin cancer. With the increasing incidence of skin cancers, low awareness among a growing population, and a lack of adequate clinical expertise and services, there is an immediate need for AI systems to assist clinicians in this domain. A large number of skin lesion datasets are available publicly, and researchers have developed AI-based image classification solutions, particularly deep learning algorithms, to distinguish malignant skin lesions fr…
Netaji Subhas University of Technology（内塔吉·苏巴斯理工大学）
Skin cancer is the most common human malignancy(American Cancer Society) which is primarily diagnosed visually, starting with an initial clinical screening and followed potentially by dermoscopic(related to skin) analysis, a biopsy and histopathological examination. Skin cancer occurs when errors (mutations) occur in the DNA of skin cells. The mutations cause the cells to grow out of control and form a mass of cancer cells. The aim of this study was to try to classify images of skin lesions with the help of convolutional neural networks. The deep neural networks show humongous potential for im…
Monash University（莫纳什大学）；Inception Institute of AI（人工智能初创研究所）；The University of Queensland（昆士兰大学）
Deep neural networks have demonstrated promising performance on image recognition tasks. However, they may heavily rely on confounding factors, using irrelevant artifacts or bias within the dataset as the cue to improve performance. When a model performs decision-making based on these spurious correlations, it can become untrustable and lead to catastrophic outcomes when deployed in the real-world scene. In this paper, we explore and try to solve this problem in the context of skin cancer diagnosis. We introduce a human-in-the-loop framework in the model training process such that users can ob…
SSN College of Engineering（SSN工程学院）；VIT University（VIT大学）
Melanoma, a malignant form of skin cancer is very threatening to life. Diagnosis of melanoma at an earlier stage is highly needed as it has a very high cure rate. Benign and malignant forms of skin cancer can be detected by analyzing the lesions present on the surface of the skin using dermoscopic images. In this work, an automated skin lesion detection system has been developed which learns the representation of the image using Google's pretrained CNN model known as Inception-v3 \cite{cnn}. After obtaining the representation vector for our input dermoscopic images we have trained two layer fe…
Faculty of Informatics, University of Debrecen（德布勒森大学信息学院）
Skin cancer is a major public health problem, with over 5 million newly diagnosed cases in the United States each year. Melanoma is the deadliest form of skin cancer, responsible for over 9,000 deaths each year. In this paper, we propose an ensemble of deep convolutional neural networks to classify dermoscopy images into three classes. To achieve the highest classification accuracy, we fuse the outputs of the softmax layers of four different neural architectures. For aggregation, we consider the individual accuracies of the networks weighted by the confidence values provided by their final sof…
In the last few years, Deep Learning (DL) has been showing superior performance in different modalities of biomedical image analysis. Several DL architectures have been proposed for classification, segmentation, and detection tasks in medical imaging and computational pathology. In this paper, we propose a new DL architecture, the NABLA-N network, with better feature fusion techniques in decoding units for dermoscopic image segmentation tasks. The NABLA-N network has several advances for segmentation tasks. First, this model ensures better feature representation for semantic segmentation with …
Federal University of Espirito Santo, UFES（圣埃斯皮里图联邦大学（UFES））
Skin cancer is one of the most common types of cancer in the world. Different computer-aided diagnosis systems have been proposed to tackle skin lesion diagnosis, most of them based in deep convolutional neural networks. However, recent advances in computer vision achieved state-of-art results in many tasks, notably Transformer-based networks. We explore and evaluate advances in computer vision architectures, training methods and multimodal feature fusion for skin lesion diagnosis task. Experiments show that PiT ($0.800 \pm 0.006$), CoaT ($0.780 \pm 0.024$) and ViT ($0.771 \pm 0.018$) backbone…
Skin cancer is one of the deadly types of cancer and is common in the world. Recently, there has been a huge jump in the rate of people getting skin cancer. For this reason, the number of studies on skin cancer classification with deep learning are increasing day by day. For the growth of work in this area, the International Skin Imaging Collaboration (ISIC) organization was established and they created an open dataset archive. In this study, images were taken from ISIC 2017 Challenge. The skin cancer images taken were preprocessed and data augmented. Later, these images were trained with tran…
In this paper, we studied extensively on different deep learning based methods to detect melanoma and skin lesion cancers. Melanoma, a form of malignant skin cancer is very threatening to health. Proper diagnosis of melanoma at an earlier stage is crucial for the success rate of complete cure. Dermoscopic images with Benign and malignant forms of skin cancer can be analyzed by computer vision system to streamline the process of skin cancer detection. In this study, we experimented with various neural networks which employ recent deep learning based models like PNASNet-5-Large, InceptionResNetV…
Lung cancer is the leading cause of cancer-related deaths in the past several years. A major challenge in lung cancer screening is the detection of lung nodules from computed tomography (CT) scans. State-of-the-art approaches in automated lung nodule classification use deep convolutional neural networks (CNNs). However, these networks require a large number of training samples to generalize well. This paper investigates the use of capsule networks (CapsNets) as an alternative to CNNs. We show that CapsNets significantly outperforms CNNs when the number of training samples is small. To increase…
United International University（联合国际大学）；The University of Western Australia（西澳大学）；Bangladesh University of Business and Technology（孟加拉国商业与技术大学）；Ahsanullah University of Science and Technology（阿萨努拉科技大学）；Charles Darwin University（查尔斯达尔文大学）
Skin cancer is a major concern to public health, accounting for one-third of the reported cancers. If not detected early, the cancer has the potential for severe consequences. Recognizing the critical need for effective skin cancer classification, we address the limitations of existing models, which are often too large to deploy in areas with limited computational resources. In response, we present a knowledge distillation based approach for creating a lightweight yet high-performing classifier. The proposed solution involves fusing three models, namely ResNet152V2, ConvNeXtBase, and ViT Base,…
Exploring the Challenge and Value of Deep Learning in Automated Skin Disease Diagnosis
Polytechnic Institute, Zhejiang University（浙江大学 polytechnic 院）；School of Social and Public Administration, East China University Of Science And Technology（东华大学社会科学与公共管理学院）；College of Science, National University of Defense Technology（国防科技大学科学学院）；School of Mathematical Sciences, Zhejiang University（浙江大学数学科学学院）
Skin cancer diagnosis using NIR spectroscopy data of skin lesions in vivo using machine learning algorithms
Federal University of Espírito Santo (UFES)（圣埃斯皮里图联邦大学）；Secretary of Health of the Espírito Santo State（圣埃斯皮里图州卫生部）；University Hospital Cassiano Antônio Moraes (HUCAM)（卡西亚诺·安东尼奥·莫赖斯大学医院）
Samsung Advanced Institute of Health Sciences and Technology（三星健康科学与技术高级研究院）；Sungkyunkwan University（成均馆大学）；Samsung Medical Center（三星医疗中心）
As skin cancer is one of the most frequent cancers globally, accurate, non-invasive dermoscopy-based diagnosis becomes essential and promising. A task of the Part 3 of the ISIC Skin Image Analysis Challenge at MICCAI 2018 is to predict seven disease classes with skin lesion images, including melanoma (MEL), melanocytic nevus (NV), basal cell carcinoma (BCC), actinic keratosis / Bowen's disease (intraepithelial carcinoma) (AKIEC), benign keratosis (solar lentigo / seborrheic keratosis / lichen planus-like keratosis) (BKL), dermatofibroma (DF) and vascular lesion (VASC) as defined by the Interna…
Melanoma is one of the most serious skin cancers that can occur in any part of the human skin. Early diagnosis of melanoma lesions will significantly increase their chances of being cured. Improving melanoma segmentation will help doctors or surgical robots remove the lesion more accurately from body parts. Recently, the learning-based segmentation methods achieved desired results in image segmentation compared to traditional algorithms. This study proposes a new approach to improve melanoma skin lesions detection and segmentation by defining a two-step pipeline based on deep learning models. …
Saliency-guided and Patch-based Mixup for Long-tailed Skin Cancer Image Classification
Southern University of Science and Technology（南方科技大学）；University of British Columbia（不列颠哥伦比亚大学）；The University of Hong Kong（香港大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
Skin cancer is by far the most common type of cancer. Early detection is the key to increase the chances for successful treatment significantly. Currently, Deep Neural Networks are the state-of-the-art results on automated skin cancer classification. To push the results further, we need to address the lack of annotated data, which is expensive and require much effort from specialists. To bypass this problem, we propose using Generative Adversarial Networks for generating realistic synthetic skin lesion images. To the best of our knowledge, our results are the first to show visually-appealing s…
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)（德国人工智能研究中心）；TU Kaiserslautern（凯泽斯劳滕工业大学）
Early detection of skin cancers like melanoma is crucial to ensure high chances of survival for patients. Clinical application of Deep Learning (DL)-based Decision Support Systems (DSS) for skin cancer screening has the potential to improve the quality of patient care. The majority of work in the medical AI community focuses on a diagnosis setting that is mainly relevant for autonomous operation. Practical decision support should, however, go beyond plain diagnosis and provide explanations. This paper provides an overview of works towards explainable, DL-based decision support in medical appli…
Federal University of Espírito Santo, UFES（圣埃斯皮里图联邦大学（UFES））
Skin cancer is a major public health problem around the world. Its early detection is very important to increase patient prognostics. However, the lack of qualified professionals and medical instruments are significant issues in this field. In this context, over the past few years, deep learning models applied to automated skin cancer detection have become a trend. In this paper, we present an overview of the recent advances reported in this field as well as a discussion about the challenges and opportunities for improvement in the current models. In addition, we also present some important as…
Skin cancer is one of the most common types of cancer around the world. For this reason, over the past years, different approaches have been proposed to assist detect it. Nonetheless, most of them are based only on dermoscopy images and do not take into account the patient clinical information. In this work, first, we present a new dataset that contains clinical images, acquired from smartphones, and patient clinical information of the skin lesions. Next, we introduce a straightforward approach to combine the clinical data and the images using different well-known deep learning models. These m…
University of Perugia（佩鲁贾大学）；University of Florence（佛罗伦萨大学）
Medical data classification is typically a challenging task due to imbalance between classes. In this paper, we propose an approach to classify dermatoscopic images from HAM10000 (Human Against Machine with 10000 training images) dataset, consisting of seven imbalanced types of skin lesions, with good precision and low resources requirements. Classification is done by using a pretrained convolutional neural network. We evaluate the accuracy and performance of the proposal and illustrate possible extensions.
