How does the accuracy of deep neural network models trained to classify clinical images of skin conditions vary across skin color? While recent studies demonstrate computer vision models can serve as a useful decision support tool in healthcare and provide dermatologist-level classification on a number of specific tasks, darker skin is underrepresented in the data. Most publicly available data sets do not include Fitzpatrick skin type labels. We annotate 16,577 clinical images sourced from two dermatology atlases with Fitzpatrick skin type labels and open-source these annotations. Based on these labels, we find that there are significantly more images of light skin types than dark skin types in this dataset. We train a deep neural network model to classify 114 skin conditions and find that the model is most accurate on skin types similar to those it was trained on. In addition, we evaluate how an algorithmic approach to identifying skin tones, individual typology angle, compares with Fitzpatrick skin type labels annotated by a team of human labelers.
University of British Columbia（不列颠哥伦比亚大学）；Simon Fraser University（西蒙弗雷泽大学）
Deep learning models have achieved great success in automating skin lesion diagnosis. However, the ethnic disparity in these models' predictions, where lesions on darker skin types are usually underrepresented and have lower diagnosis accuracy, receives little attention. In this paper, we propose FairDisCo, a disentanglement deep learning framework with contrastive learning that utilizes an additional network branch to remove sensitive attributes, i.e. skin-type information from representations for fairness and another contrastive branch to enhance feature extraction. We compare FairDisCo to t…
More than 3 billion people lack access to care for skin disease. AI diagnostic tools may aid in early skin cancer detection; however most models have not been assessed on images of diverse skin tones or uncommon diseases. To address this, we curated the Diverse Dermatology Images (DDI) dataset - the first publicly available, pathologically confirmed images featuring diverse skin tones. We show that state-of-the-art dermatology AI models perform substantially worse on DDI, with ROC-AUC dropping 29-40 percent compared to the models' original results. We find that dark skin tones and uncommon dis…
SkinCaRe: A Multimodal Dermatology Dataset Annotated with Medical Caption and Chain-of-Thought Reasoning
School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK–Shenzhen)（数据科学学院，香港中文大学（深圳））；Computer Science Program, CEMSE Division, King Abdullah University of Science and Technology (KAUST)（计算机科学项目，科学与工程学院，国王 Abdullah 科学技术大学）；Center of Excellence on Smart Health, KAUST（智能健康卓越中心，国王 Abdullah 科学技术大学）；Center of Excellence for Generative AI, KAUST（生成式人工智能卓越中心，国王 Abdullah 科学技术大学）；Department of Dermatology, Beijing AnZhen Hospital, Capital Medical University（皮肤科，北京安贞医院，首都医科大学）；Department of Dermatology, Tianjin Institute of Integrative Dermatology, Tianjin Academy of Traditional Chinese Medicine Affiliated Hospital（皮肤科，天津整合皮肤科研究院，天津中医药大学附属医院）；Department of Dermatology, Beijing Aerospace General Hospital（皮肤科，北京航天总医院）；Department of Dermatology, The First Affiliated Hospital, Shantou University Medical College（皮肤科，汕头大学医学院第一附属医院）；DermAssure, LLC（DermAssure 公司）；School of Medicine, New York Medical College（医学院，纽约医学院）；Capital Medical University（首都医科大学）；Department of Dermatology, Second Hospital of Jilin University（皮肤科，吉林大学第二医院）；Emergency Critical Care Center, Beijing AnZhen Hospital, Capital Medical University（急诊重症中心，北京安贞医院，首都医科大学）
DermaSynth: Rich Synthetic Image-Text Pairs Using Open Access Dermatology Datasets
Imperial College London（帝国理工学院）；Yildiz Technical University（伊尔德兹技术大学）；Usak Research and Training Hospital（乌萨克研究与培训医院）；Istanbul Medeniyet University（伊斯坦布尔梅德尼耶特大学）；Universität der Bundeswehr München（慕尼黑联邦国防军大学）；Istanbul Medicana Atakoy Hospital（伊斯坦布尔 Medicana Atakoy 医院）
Stanford School of Medicine（斯坦福大学医学院）；Stanford University（斯坦福大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Keio University School of Medicine（庆应义塾大学医学院）；Chan-Zuckerberg Biohub（陈-扎克伯格生物中心）
Access to dermatological care is a major issue, with an estimated 3 billion people lacking access to care globally. Artificial intelligence (AI) may aid in triaging skin diseases. However, most AI models have not been rigorously assessed on images of diverse skin tones or uncommon diseases. To ascertain potential biases in algorithm performance in this context, we curated the Diverse Dermatology Images (DDI) dataset-the first publicly available, expertly curated, and pathologically confirmed image dataset with diverse skin tones. Using this dataset of 656 images, we show that state-of-the-art …
For the deployment of artificial intelligence (AI) in high-risk settings, such as healthcare, methods that provide interpretability/explainability or allow fine-grained error analysis are critical. Many recent methods for interpretability/explainability and fine-grained error analysis use concepts, which are meta-labels that are semantically meaningful to humans. However, there are only a few datasets that include concept-level meta-labels and most of these meta-labels are relevant for natural images that do not require domain expertise. Densely annotated datasets in medicine focused on meta-l…
DermETAS-SNA LLM: A Dermatology Focused Evolutionary Transformer Architecture Search with StackNet Augmented LLM Assistant
Data Science Program, Worcester Polytechnic Institute, Massachusetts, USA（沃斯特理工学院数据科学项目，马萨诸塞州，美国）；Computer Science Department, Worcester Polytechnic Institute, Massachusetts, USA（沃斯特理工学院计算机科学系，马萨诸塞州，美国）；PASE Advisory Group, New Jersey, USA（PASE顾问小组，新泽西州，美国）
DermINO: Hybrid Pretraining for a Versatile Dermatology Foundation Model
Microsoft Research Asia（微软亚洲研究院）；Peking University China-Japan Friendship School of Clinical Medicine（北京大学中国日友好临床医学学院）；Scientific-skincare Innovation Alliance (SIA)（科学护肤创新联盟（SIA））
Carnegie Mellon University Africa（卡内基梅隆大学非洲分校）；IBM Research（IBM研究院）
Recent advances in computer vision and deep learning have led to breakthroughs in the development of automated skin image analysis. In particular, skin cancer classification models have achieved performance higher than trained expert dermatologists. However, no attempt has been made to evaluate the consistency in performance of machine learning models across populations with varying skin tones. In this paper, we present an approach to estimate skin tone in benchmark skin disease datasets, and investigate whether model performance is dependent on this measure. Specifically, we use individual ty…
Harvard Medical School（哈佛医学院）；UT Austin Dell Medical School（德克萨斯大学奥斯汀分校戴尔医学院）
Dermatological classification algorithms developed without sufficiently diverse training data may generalize poorly across populations. While intentional data collection and annotation offer the best means for improving representation, new computational approaches for generating training data may also aid in mitigating the effects of sampling bias. In this paper, we show that DALL$\cdot$E 2, a large-scale text-to-image diffusion model, can produce photorealistic images of skin disease across skin types. Using the Fitzpatrick 17k dataset as a benchmark, we demonstrate that augmenting training d…
DermaCon-IN: A Multi-concept Annotated Dermatological Image Dataset of Indian Skin Disorders for Clinical AI Research
Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, India（人工智能系，印度理工学院海得拉巴学院，印度）；Indian Navy（印度海军）；Department of Dermatology, S R Patil Medical College, India（皮肤科系，S R Patil医学院，印度）；Department of Dermatology, S Nijalingappa Medical College, India（皮肤科系，S Nijalingappa医学院，印度）；Department of Dermatology, Sri Chamundeshwari Medical College, Hospital & Research, India（皮肤科系，Sri Chamundeshwari医学院，医院及研究，印度）；Indian Institute of Technology Hyderabad, India（印度理工学院海得拉巴学院，印度）
Stanford School of Medicine（斯坦福大学医学院）；University of Pennsylvania（宾夕法尼亚大学）
Artificial intelligence (AI) algorithms using deep learning have advanced the classification of skin disease images; however these algorithms have been mostly applied "in silico" and not validated clinically. Most dermatology AI algorithms perform binary classification tasks (e.g. malignancy versus benign lesions), but this task is not representative of dermatologists' diagnostic range. The American Academy of Dermatology Task Force on Augmented Intelligence published a position statement emphasizing the importance of clinical validation to create human-computer synergy, termed augmented intel…
Towards Trustworthy Dermatology MLLMs: A Benchmark and Multimodal Evaluator for Diagnostic Narratives
School of Data Science, The Chinese University of Hong Kong, Shenzhen（数据科学学院，香港中文大学（深圳））；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Department of Dermatology, The First Affiliated Hospital, Shantou University Medical College（皮肤科，汕头大学医学院第一附属医院）
DermAI: Clinical dermatology acquisition through quality-driven image collection for AI classification in mobile
Centro de Informática, Universidade Federal de Pernambuco, Brazil（巴西佩纳布卢克联邦大学计算机中心）；Hospital das Clínicas, Universidade Federal de Pernambuco, Brazil（巴西佩纳布卢克联邦大学临床医院）
DermaBench: A Clinician-Annotated Benchmark Dataset for Dermatology Visual Question Answering and Reasoning
Imperial College London（帝国理工学院伦敦分校）；Istanbul Medeniyet University（伊斯坦布尔医学大学）；Usak Research and Training Hospital（Usak研究与培训医院）；Istanbul Research and Training Hospital（伊斯坦布尔研究与培训医院）；Ipswich Hospital（伊普斯韦奇医院）；Medicana Atakoy Hospital（Medicana阿塔基医院）
The Impact of Skin Tone Label Granularity on the Performance and Fairness of AI Based Dermatology Image Classification Models
School of Biomedical Engineering and Imaging Sciences, King's College London, UK（生物医学工程与成像科学学院，伦敦国王学院，英国）；Dermatology Department, Kings College Hospital NHS Foundation Trust, London, UK（皮肤科部门，国王学院医院国家健康服务信托基金，伦敦，英国）
Colorimeter-Supervised Skin Tone Estimation from Dermatoscopic Images for Fairness Auditing
Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, Josip Juraj Strossmayer University of Osijek（奥西杰克大学电子工程、计算机科学与信息科技学院）
MIT Media Lab（麻省理工学院媒体实验室）；Stanford University（斯坦福大学）；Northeast Dermatology Associates（东北皮肤病协会）；Banner Health（班纳健康）
While artificial intelligence (AI) holds promise for supporting healthcare providers and improving the accuracy of medical diagnoses, a lack of transparency in the composition of datasets exposes AI models to the possibility of unintentional and avoidable mistakes. In particular, public and private image datasets of dermatological conditions rarely include information on skin color. As a start towards increasing transparency, AI researchers have appropriated the use of the Fitzpatrick skin type (FST) from a measure of patient photosensitivity to a measure for estimating skin tone in algorithmi…
Florida Institute of Technology（佛罗里达理工学院）；University of Notre Dame（圣母大学）
News reports have suggested that darker skin tone causes an increase in face recognition errors. The Fitzpatrick scale is widely used in dermatology to classify sensitivity to sun exposure and skin tone. In this paper, we analyze a set of manual Fitzpatrick skin type assignments and also employ the individual typology angle to automatically estimate the skin tone from face images. The set of manual skin tone rating experiments shows that there are inconsistencies between human raters that are difficult to eliminate. Efforts to automate skin tone rating suggest that it is particularly challengi…
An analysis of data variation and bias in image-based dermatological datasets for machine learning classification
Centro de Informática, Universidade Federal de Pernambuco (UFPE)（伯南布哥联邦大学信息中心）；Ambulatório de Dermatologia, Hospital das Clinicas - UFPE（伯南布哥联邦大学临床医院皮肤科门诊）
University of Basel（巴塞尔大学）；Lucerne University of Applied Sciences and Arts（卢塞恩应用科学大学）；University Hospital of Basel（巴塞尔大学医院）；Northwestern University（西北大学）；Northeast Dermatology Associates（东北皮肤科诊所）；Medical University of Vienna（维也纳医科大学）；Banner Health（Banner健康系统）
A Multimodal Vision Foundation Model for Clinical Dermatology
Monash University（莫纳什大学）；The University of Queensland（昆士兰大学）；Medical University of Vienna（维也纳医科大学）；Alfred Hospital（阿尔弗雷德医院）；Princess Alexandra Hospital（亚历山德拉公主医院）；NVIDIA AI Technology Center（英伟达人工智能技术中心）；University of Florence（佛罗伦萨大学）；Melanoma Institute Australia（澳大利亚黑色素瘤研究所）；Royal Prince Alfred Hospital（阿尔弗雷德王子医院）；Hospital General Universitario de Alicante（阿利坎特大学综合医院）
Are generative models fair? A study of racial bias in dermatological image generation
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València（瓦伦西亚理工大学 以人为本技术大学研究所）；Technical University of Denmark（丹麦技术大学）
Recent advances in deep learning have led to breakthroughs in the development of automated skin disease classification. As we observe an increasing interest in these models in the dermatology space, it is crucial to address aspects such as the robustness towards input data distribution shifts. Current skin disease models could make incorrect inferences for test samples from different hardware devices and clinical settings or unknown disease samples, which are out-of-distribution (OOD) from the training samples. To this end, we propose a simple yet effective approach that detect these OOD sampl…
UltraLBM-UNet: Ultralight Bidirectional Mamba-based Model for Skin Lesion Segmentation
Data Science Institute, Vanderbilt University（维斯尼尔大学数据科学学院）；College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）
A Vision-Language Foundation Model for Zero-shot Clinical Collaboration and Automated Concept Discovery in Dermatology
AIM for Health Lab（AIM for Health实验室）；Faculty of Information Technology, Monash University（信息技术学院，墨尔本大学）；Department of Dermatology, Medical University of Vienna（皮肤科，维也纳医学大学）；Faculty of Engineering, Monash University（工程学院，墨尔本大学）；Institute of Ophthalmology, University College London（眼科研究所，伦敦大学学院）；Dermatology Department. Fundacion Hospital 12 de Octubre（皮肤科部门，12月12日基金会医院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（数据科学学院，香港中文大学（深圳））；Frazer Institute, The University of Queensland（弗拉泽研究所，昆士兰大学）；Dermatology Research Centre, Brisbane, Australia（皮肤科研究中心，澳大利亚布里斯班）；Department of Medical and Cardiovascular Sciences, Sapienza University of Rome（医学与心血管科学系，罗马萨皮恩扎大学）；Victorian Melanoma Service, Alfred Care Group, Bayside Health, Melbourne, Australia（维多利亚黑色素瘤服务，阿尔弗雷德护理集团，墨尔本布里斯班健康中心，澳大利亚）；SkIIN Discovery Program, Monash University（SkIIN发现计划，墨尔本大学）；Claude Bernard Lyon-1 University（克劳德·伯纳德里昂-1大学）；Centre Léon Berard, Lyon, France（莱昂·贝拉德中心，法国里昂）；Dermatology department, Hôpital Lyon Sud, Hospices Civils de Lyon, Lyons, France（皮肤科部门，里昂南医院，里昂民事医院，法国里昂）；Cancer Research Center of Lyon, Lyons, France（里昂癌症研究中心，法国里昂）；eResearch Centre, Monash University（eResearch研究中心，墨尔本大学）；NVIDIA AI Techno（NVIDIA AI技术）
Technical University of Denmark（丹麦技术大学）；Medable A/S（Medable公司）；University of Copenhagen（哥本哈根大学）；Rigshospitalet, Copenhagen University Hospital（哥本哈根大学医院国立医院）
In recent years, large strides have been taken in developing machine learning methods for dermatological applications, supported in part by the success of deep learning (DL). To date, diagnosing diseases from images is one of the most explored applications of DL within dermatology. Convolutional neural networks (ConvNets) are the most common (DL) method in medical imaging due to their training efficiency and accuracy, although they are often described as black boxes because of their limited explainability. One popular way to obtain insight into a ConvNet's decision mechanism is gradient class …
Google Health（谷歌健康）；Google Research（谷歌研究院）；DeepMind（深度思维）
We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier. We frame this task as an out-of-distribution (OOD) detection problem. Our novel approach, hierarchical outlier detection (HOD) assigns multiple abstention classes for each training outlier class and jointly performs a coarse classification of inliers vs. outliers, along with fine-grained classification of the individual classes. We demonstrate the effectiveness of the HOD lo…
DermAgent: A Self-Reflective Agentic System for Dermatological Image Analysis with Multi-Tool Reasoning and Traceable Decision-Making
AIM for Health Lab, Faculty of Information Technology, Monash University, Melbourne, Australia（健康人工智能实验室，信息科技学院，墨尔本大学，澳大利亚）；Faculty of Information Technology, Monash University, Melbourne, Australia（信息科技学院，墨尔本大学，澳大利亚）；University College London, Institute of Ophthalmology, London, United Kingdom（伦敦大学学院，眼科研究所，英国）
