We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images, which can be indistinctly acquired via smartphone or dermoscopy capture. Additionally, we do not assume that images contain single lesions, thus the framework supports both focal or wide-field images. Specifically, we propose a two-stage approach in which we first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy that can be used for high-level screening processes. Further, we consider augmenting the proposed approach with clinical covariates (from electronic health records) and publicly available data (the ISIC dataset). Comprehensive experiments validated on an independent test dataset demonstrate that i) the proposed approach outperforms alternative model architectures; ii) the model based on images outperforms a pure clinical model by a large margin, and the combination of images and clinical data does not significantly improves over the image-only model; and iii) the proposed framework offers comparable performance in terms of malignancy classification relative to three board certified dermatologists with different levels of experience.
Guangdong Provincial People’s Hospital（广东省人民医院）；University of Notre Dame（圣母大学）；University of Pittsburgh（匹兹堡大学）
Many works have shown that deep learning-based medical image classification models can exhibit bias toward certain demographic attributes like race, gender, and age. Existing bias mitigation methods primarily focus on learning debiased models, which may not necessarily guarantee all sensitive information can be removed and usually comes with considerable accuracy degradation on both privileged and unprivileged groups. To tackle this issue, we propose a method, FairPrune, that achieves fairness by pruning. Conventionally, pruning is used to reduce the model size for efficient inference. However…
Deep learning has played a major role in the interpretation of dermoscopic images for detecting skin defects and abnormalities. However, current deep learning solutions for dermatological lesion analysis are typically limited in providing probabilistic predictions which highlights the importance of concerning uncertainties. This concept of uncertainty can provide a confidence level for each feature which prevents overconfident predictions with poor generalization on unseen data. In this paper, we propose an overall framework that jointly considers dermatological classification and uncertainty …
An analysis of data variation and bias in image-based dermatological datasets for machine learning classification
Centro de Informática, Universidade Federal de Pernambuco (UFPE)（伯南布哥联邦大学信息中心）；Ambulatório de Dermatologia, Hospital das Clinicas - UFPE（伯南布哥联邦大学临床医院皮肤科门诊）
Classification of malignancy for breast cancer and other cancer types is usually tackled as an object detection problem: Individual lesions are first localized and then classified with respect to malignancy. However, the drawback of this approach is that abstract features incorporating several lesions and areas that are not labelled as a lesion but contain global medically relevant information are thus disregarded: especially for dynamic contrast-enhanced breast MRI, criteria such as background parenchymal enhancement and location within the breast are important for diagnosis and cannot be cap…
Skin conditions are a global health concern, ranking the fourth highest cause of nonfatal disease burden when measured as years lost due to disability. As diagnosing, or classifying, skin diseases can help determine effective treatment, dermatologists have extensively researched how to diagnose conditions from a patient's history and the lesion's visual appearance. Computer vision researchers are attempting to encode this diagnostic ability into machines, and several recent studies report machine level performance comparable with dermatologists. This report reviews machine approaches to classi…
Skin cancer is the most common malignancy in the world. Automated skin cancer detection would significantly improve early detection rates and prevent deaths. To help with this aim, a number of datasets have been released which can be used to train Deep Learning systems - these have produced impressive results for classification. However, this only works for the classes they are trained on whilst they are incapable of identifying skin lesions from previously unseen classes, making them unconducive for clinical use. We could look to massively increase the datasets by including all possible skin …
DermINO: Hybrid Pretraining for a Versatile Dermatology Foundation Model
Microsoft Research Asia（微软亚洲研究院）；Peking University China-Japan Friendship School of Clinical Medicine（北京大学中国日友好临床医学学院）；Scientific-skincare Innovation Alliance (SIA)（科学护肤创新联盟（SIA））
Nanyang Technological University（南洋理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Vancouver Coast Health Research Institute（温哥华海岸健康研究所）；BC Cancer（不列颠哥伦比亚癌症中心）
Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Over the last few years, computer-aided diagnosis has been rapidly developed and make great progress in healthcare and medical practices due to the advances in artificial intelligence. However, most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. In this case, knowledge distillation (KD) has been proven as an efficient tool to help improve the adaptability o…
Technische Universit\"at M\"unchen, Image-Based Biomedical Modeling Group, Arccisstrasse 21, 80333 Munich Technische Universit\"at M\"unchen, Institute for diagnostic；interventional Radiology, Ismaninger Str. 22, 81675 Munich + Ludwig-Maximilians-Universit\"at, German Center for Vertigo
Automatic non-invasive assessment of hepatocellular carcinoma (HCC) malignancy has the potential to substantially enhance tumor treatment strategies for HCC patients. In this work we present a novel framework to automatically characterize the malignancy of HCC lesions from DWI images. We predict HCC malignancy in two steps: As a first step we automatically segment HCC tumor lesions using cascaded fully convolutional neural networks (CFCN). A 3D neural network (SurvivalNet) then predicts the HCC lesions' malignancy from the HCC tumor segmentation. We formulate this task as a classification prob…
The University of Adelaide（阿德莱德大学）；Queensland University of Technology（昆士兰科技大学）；Instituto Superior Tecnico（里斯本高等理工学院）
We propose a new method for breast cancer screening from DCE-MRI based on a post-hoc approach that is trained using weakly annotated data (i.e., labels are available only at the image level without any lesion delineation). Our proposed post-hoc method automatically diagnosis the whole volume and, for positive cases, it localizes the malignant lesions that led to such diagnosis. Conversely, traditional approaches follow a pre-hoc approach that initially localises suspicious areas that are subsequently classified to establish the breast malignancy -- this approach is trained using strongly annot…
Ultrasound-based detection and malignancy prediction of breast lesions eligible for biopsy: A multi-center clinical-scenario study using nomograms, large language models, and radiologist evaluation
Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）；Urmia University of Medical Science（乌尔米耶医科大学）；University of Health Sciences（健康科学大学）；Kartal Dr. Lütfi Kırdar City Hospital（卡尔塔尔·吕特菲·克尔达尔市医院）；University of Padova（帕多瓦大学）；Universal Scientific Education and Research Network (USERN)（全球科学教育与研究网络（USERN））；Tehran University of Medical Sciences（德黑兰医科大学）；Danube Private University（多瑙河私立大学）；Austrian Center for Medical Innovation and Technology (ACMIT)（奥地利医学创新与技术中心（ACMIT））；University of Southern Queensland（南昆士兰大学）
Ensemble Deep Learning and LLM-Assisted Reporting for Automated Skin Lesion Diagnosis
Institute of Computer Science The University of Agriculture Peshawar, Pakistan（计算机科学研究所 伊斯兰农业大学 薩瓦爾，巴基斯坦）；Department of Computer Science Islamia College Peshawar Pakistan（计算机科学系 伊斯兰学院 薩瓦爾，巴基斯坦）；Department of Software Engineering Islamia College Peshawar Pakistan（软件工程系 伊斯兰学院 薩瓦爾，巴基斯坦）
German Research Center for Artificial Intelligence (DFKI) GmbH（德国人工智能研究中心）；Technical University Kaiserslautern（凯泽斯劳滕工业大学）；University Hospital Münster（明斯特大学医院）；University Hospital of Düsseldorf（杜塞尔多夫大学医院）；School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST)（国立科技大学电气工程与计算机科学学院）；Deep Learning Laboratory, National Center of Artificial Intelligence（国家人工智能中心深度学习实验室）
One principal impediment in the successful deployment of AI-based Computer-Aided Diagnosis (CAD) systems in clinical workflows is their lack of transparent decision making. Although commonly used eXplainable AI methods provide some insight into opaque algorithms, such explanations are usually convoluted and not readily comprehensible except by highly trained experts. The explanation of decisions regarding the malignancy of skin lesions from dermoscopic images demands particular clarity, as the underlying medical problem definition is itself ambiguous. This work presents ExAID (Explainable AI f…
Visual Bias and Interpretability in Deep Learning for Dermatological Image Analysis
Department of Computer Science and Engineering, European University of Bangladesh（计算机科学与工程系，孟加拉国欧洲大学）；Department of Computer Science and Engineering, BRAC University（计算机科学与工程系，BRAC大学）；Department of Information Systems, University of Maryland, Baltimore County（信息系统系，马里兰大学巴尔的摩县分校）
Stanford School of Medicine（斯坦福大学医学院）；Stanford University（斯坦福大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Keio University School of Medicine（庆应义塾大学医学院）；Chan-Zuckerberg Biohub（陈-扎克伯格生物中心）
Access to dermatological care is a major issue, with an estimated 3 billion people lacking access to care globally. Artificial intelligence (AI) may aid in triaging skin diseases. However, most AI models have not been rigorously assessed on images of diverse skin tones or uncommon diseases. To ascertain potential biases in algorithm performance in this context, we curated the Diverse Dermatology Images (DDI) dataset-the first publicly available, expertly curated, and pathologically confirmed image dataset with diverse skin tones. Using this dataset of 656 images, we show that state-of-the-art …
Data scarcity and class imbalance are two fundamental challenges in many machine learning applications to healthcare. Breast cancer classification in mammography exemplifies these challenges, with a malignancy rate of around 0.5% in a screening population, which is compounded by the relatively small size of lesions (~1% of the image) in malignant cases. Simultaneously, the prevalence of screening mammography creates a potential abundance of non-cancer exams to use for training. Altogether, these characteristics lead to overfitting on cancer cases, while under-utilizing non-cancer data. Here, w…
MasHeNe: A Benchmark for Head and Neck CT Mass Segmentation using Window-Enhanced Mamba with Frequency-Domain Integration
University of Science, VNU-HCM, Ho Chi Minh City, Vietnam（越南胡志明市VNU-HCM大学科学学院）；Vietnam National University, Ho Chi Minh City, Vietnam（越南国家大学）；Thong Nhat Hospital, Ho Chi Minh City, Vietnam（胡志明市通那特医院）；University of Social Sciences and Humanities, VNU-HCM, Ho Chi Minh City, Vietnam（越南胡志明市VNU-HCM大学社会科学与人文学院）；Dublin City University, Dublin, Ireland（都柏林城市大学）；University of Health Sciences, VNU-HCM, Ho Chi Minh City, Vietnam（越南胡志明市VNU-HCM大学健康科学学院）；John von Neumann Institute, VNU-HCM, Ho Chi Minh City, Vietnam（约翰·冯·诺依曼研究所）
While deep learning based approaches have demonstrated expert-level performance in dermatological diagnosis tasks, they have also been shown to exhibit biases toward certain demographic attributes, particularly skin types (e.g., light versus dark), a fairness concern that must be addressed. We propose CIRCLe, a skin color invariant deep representation learning method for improving fairness in skin lesion classification. CIRCLe is trained to classify images by utilizing a regularization loss that encourages images with the same diagnosis but different skin types to have similar latent represent…
Skin cancer, the most common human malignancy, is primarily diagnosed visually by physicians [1]. Classification with an automated method like CNN [2, 3] shows potential for challenging tasks [1]. By now, the deep convolutional neural networks are on par with human dermatologist [1]. This abstract is dedicated on developing a Deep Learning method for ISIC [5] 2017 Skin Lesion Detection Competition hosted at [6] to classify the dermatology pictures, which is aimed at improving the diagnostic accuracy rate and general level of the human health. The challenge falls into three sub-challenges, incl…
Isfahan University of Technology（伊斯法罕理工大学）；University of Michigan（密歇根大学）
Melanoma is amongst most aggressive types of cancer. However, it is highly curable if detected in its early stages. Prescreening of suspicious moles and lesions for malignancy is of great importance. Detection can be done by images captured by standard cameras, which are more preferable due to low cost and availability. One important step in computerized evaluation of skin lesions is accurate detection of lesion region, i.e. segmentation of an image into two regions as lesion and normal skin. Accurate segmentation can be challenging due to burdens such as illumination variation and low contras…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Novartis Pharmaceuticals（诺华制药）；University of California, Irvine（加州大学欧文分校）；Wolfram Research（沃尔夫勒姆研究公司）
Skin cancer, the most commonly found human malignancy, is primarily diagnosed visually via dermoscopic analysis, biopsy, and histopathological examination. However, unlike other types of cancer, automated image classification of skin lesions is deemed more challenging due to the irregularity and variability in the lesions' appearances. In this work, we propose an adaptation of the Neural Style Transfer (NST) as a novel image pre-processing step for skin lesion classification problems. We represent each dermoscopic image as the style image and transfer the style of the lesion onto a homogeneous…
Melanoma is considered to be the most aggressive form of skin cancer. Due to the similar shape of malignant and benign cancerous lesions, doctors spend considerably more time when diagnosing these findings. At present, the evaluation of malignancy is performed primarily by invasive histological examination of the suspicious lesion. Developing an accurate classifier for early and efficient detection can minimize and monitor the harmful effects of skin cancer and increase patient survival rates. This paper proposes a multi-class classification task using the CoAtNet architecture, a hybrid model …
Skin cancer is considered to be the most common human malignancy. Around 5 million new cases of skin cancer are recorded in the United States annually. Early identification and evaluation of skin lesions is of great clinical significance, but the disproportionate dermatologist-patient ratio poses significant problem in most developing nations. Therefore a deep learning based architecture, known as SkiNet, is proposed with an objective to provide faster screening solution and assistance to newly trained physicians in the clinical diagnosis process. The main motive behind Skinet's design and dev…
Department of Epidemiology and Biostatistics, College of Public Health, Zhengzhou University（郑州大学公共卫生学院流行病学与生物统计学系）；Victorian Melanoma Service, Alfred Health（阿尔弗雷德健康服务中心维多利亚黑色素瘤中心）；eResearch Centre, Monash University（莫纳什大学电子研究中心）
Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transfo…
Radboud University Medical Center（拉德堡德大学医学中心）；Delft University of Technology（代尔夫特理工大学）
Computer-aided detection or decision support systems aim to improve breast cancer screening programs by helping radiologists to evaluate digital mammography (DM) exams. Commonly such methods proceed in two steps: selection of candidate regions for malignancy, and later classification as either malignant or not. In this study, we present a candidate detection method based on deep learning to automatically detect and additionally segment soft tissue lesions in DM. A database of DM exams (mostly bilateral and two views) was collected from our institutional archive. In total, 7196 DM exams (28294 …
Sun Yat-sen University（中山大学）；The First Affiliated Hospital, Sun Yat-sen University（中山大学附属第一医院）；Infervision Inc.（推想医疗科技股份有限公司）
Hepatocellular carcinoma (HCC) is the second most frequent cause of malignancy-related death and is one of the diseases with the highest incidence in the world. Because the liver is the only organ in the human body that is supplied by two major vessels: the hepatic artery and the portal vein, various types of malignant tumors can spread from other organs to the liver. And due to the liver masses' heterogeneous and diffusive shape, the tumor lesions are very difficult to be recognized, thus automatic lesion detection is necessary for the doctors with huge workloads. To assist doctors, this work…
German Cancer Research Center（德国癌症研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；DeepMind（深度思维）
For the task of concurrently detecting and categorizing objects, the medical imaging community commonly adopts methods developed on natural images. Current state-of-the-art object detectors are comprised of two stages: the first stage generates region proposals, the second stage subsequently categorizes them. Unlike in natural images, however, for anatomical structures of interest such as tumors, the appearance in the image (e.g., scale or intensity) links to a malignancy grade that lies on a continuous ordinal scale. While classification models discard this ordinal relation between grades by …
Hide-and-Seek Attribution: Weakly Supervised Segmentation of Vertebral Metastases in CT
Institute of Neuroradiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑工业大学大学医院神经放射学研究所，医学与健康学院，慕尼黑工业大学）；Chair for AI in Healthcare and Medicine, TUM and TUM University Hospital（慕尼黑工业大学及大学医院医疗与医学人工智能教席）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Institute of Diagnostic and Interventional Radiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑工业大学大学医院诊断与介入放射学研究所，医学与健康学院，慕尼黑工业大学）；Department of Neuroradiology, University Hospital Munich (LMU)（慕尼黑大学医院神经放射学系）；Department of Quantitative Biomedicine, University of Zurich（苏黎世大学定量生物医学系）；Department of Computing, Imperial College London（伦敦帝国理工学院计算系）；Department of Diagnostic and Interventional Radiology, University Hospital Frankfurt（法兰克福大学医院诊断与介入放射学系）；Department of Orthopedics and Sports Orthopedics, TUM University Hospital, School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑工业大学大学医院骨科与运动骨科，医学与健康学院，慕尼黑工业大学）
Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization
University of Technology Sydney（悉尼科技大学）；Saigon Precision Medicine Research Center（西贡精准医学研究中心）；College of Medicine, Taipei Medical University（台北医学大学医学院）；Le Qui Don Technical University（黎文谍技术大学）；National Central University（中央大学）；AIBioMed Research Group, Taipei Medical University（台北医学大学AIBioMed研究组）
From Density to Biopsy Decisions and Malignancy Prediction: A Benchmark Study of Multimodal Large Language Models Against Radiologists in Digital and Contrast-Enhanced Mammography
Danube Private University（多瑙河私立大学）；Urmia University of Medical Science（乌尔米耶医科大学）；Yeditepe University（耶迪特佩大学）；Kartal Dr. LÜtfi Kırdar City Hospital（卡尔塔尔·卢特菲·克尔达尔博士市立医院）；Iran University of Medical Sciences（伊朗医科大学）；University of Padova（帕多瓦大学）；Bahçeşehir University（巴赫切谢希尔大学）；Mooney's Bay Pain Clinic（穆尼湾疼痛诊所）；Arden University（雅顿大学）；University Hospital Cologne（科隆大学医院）；University of Southern Queensland（南昆士兰大学）；Austrian Center for Medical Innovation and Technology（奥地利医学创新与技术中心）
