Deep learning has significantly advanced automatic medical diagnostics and released the occupation of human resources to reduce clinical pressure, yet the persistent challenge of data scarcity in this area hampers its further improvements and applications. To address this gap, we introduce a novel ensemble framework called `Efficient Transfer and Self-supervised Learning based Ensemble Framework' (ETSEF). ETSEF leverages features from multiple pre-trained deep learning models to efficiently learn powerful representations from a limited number of data samples. To the best of our knowledge, ETSEF is the first strategy that combines two pre-training methodologies (Transfer Learning and Self-supervised Learning) with ensemble learning approaches. Various data enhancement techniques, including data augmentation, feature fusion, feature selection, and decision fusion, have also been deployed to maximise the efficiency and robustness of the ETSEF model. Five independent medical imaging tasks, including endoscopy, breast cancer, monkeypox, brain tumour, and glaucoma detection, were tested to demonstrate ETSEF's effectiveness and robustness. Facing limited sample numbers and challenging medical tasks, ETSEF has proved its effectiveness by improving diagnostics accuracies from 10\% to 13.3\% when compared to strong ensemble baseline models and up to 14.4\% improvements compared with published state-of-the-art methods. Moreover, we emphasise the robustness and trustworthiness of the ETSEF method through various vision-explainable artificial intelligence techniques, including Grad-CAM, SHAP, and t-SNE. Compared to those large-scale deep learning models, ETSEF can be deployed flexibly and maintain superior performance for challenging medical imaging tasks, showing the potential to be applied to more areas that lack training data
EPFL（洛桑联邦理工学院）；IBM Research Zürich（IBM苏黎世研究院）；ETH Zürich（苏黎世联邦理工学院）；National Cancer Institute - IRCCS-Fondazione Pascale（国家癌症研究所 - IRCCS帕斯卡尔基金会）
Explainability of machine learning (ML) techniques in digital pathology (DP) is of great significance to facilitate their wide adoption in clinics. Recently, graph techniques encoding relevant biological entities have been employed to represent and assess DP images. Such paradigm shift from pixel-wise to entity-wise analysis provides more control over concept representation. In this paper, we introduce a post-hoc explainer to derive compact per-instance explanations emphasizing diagnostically important entities in the graph. Although we focus our analyses to cells and cellular interactions in …
SilVar-Med: A Speech-Driven Visual Language Model for Explainable Abnormality Detection in Medical Imaging
Florida Institute of Technology（佛罗里达理工学院）；Knovel Engineering Lab（诺维尔工程实验室）；Vietnam Military Medical University（越南军事医科大学）；Military Central Hospital（108陆军中央医院）；Can Tho University of Medicine and Pharmacy（芹苴医药大学）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Deep learning methods have been very effective for a variety of medical diagnostic tasks and has even beaten human experts on some of those. However, the black-box nature of the algorithms has restricted clinical use. Recent explainability studies aim to show the features that influence the decision of a model the most. The majority of literature reviews of this area have focused on taxonomy, ethics, and the need for explanations. A review of the current applications of explainable deep learning for different medical imaging tasks is presented here. The various approaches, challenges for clini…
A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis
Zhejiang University-University of Illinois at Urbana-Champaign Institute（浙江大学伊利诺伊大学厄巴纳香槟校区联合学院）；Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）；Angelalign Inc.（时代天使公司）
MedicalPatchNet: A Patch-Based Self-Explainable AI Architecture for Chest X-ray Classification
Department of Diagnostic and Interventional Radiology of the University Hospital RWTH Aachen（鲁尔大学亚琛医院诊断与介入放射科）；Else Kroener Fresenius Center for Digital Health, Technical University Dresden（德累斯顿技术大学埃勒克罗纳弗雷斯内尔数字健康中心）；Department of Medicine III, University Hospital RWTH Aachen（鲁尔大学亚琛医院第三医学部）；Pathology & Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds（利兹大学圣詹姆斯医院医学研究所以及病理学与数据科学）；Department of Medicine I, University Hospital Dresden（德累斯顿大学第一医学部）；Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg（海德堡大学国家肿瘤疾病中心（NCT）肿瘤医学部）
Indian Institute of Technology (IIT) Ropar（印度理工学院罗巴尔分校）；Monash University（莫纳什大学）；University of Canberra（堪培拉大学）；Inception Institute of Artificial Intelligence（因塞普申人工智能研究院）
We present a novel framework for explainable labeling and interpretation of medical images. Medical images require specialized professionals for interpretation, and are explained (typically) via elaborate textual reports. Different from prior methods that focus on medical report generation from images or vice-versa, we novelly generate congruent image--report pairs employing a cyclic-Generative Adversarial Network (cycleGAN); thereby, the generated report will adequately explain a medical image, while a report-generated image that effectively characterizes the text visually should (sufficientl…
University of Turin（都灵大学）；Azienda Ospedaliera Città della Salute e della Scienza Presidio Molinette（健康与科学城医院莫利内特分院）；AOU San Luigi Gonzaga（圣路易吉·贡扎加大学医院）
Early screening of patients is a critical issue in order to assess immediate and fast responses against the spread of COVID-19. The use of nasopharyngeal swabs has been considered the most viable approach; however, the result is not immediate or, in the case of fast exams, sufficiently accurate. Using Chest X-Ray (CXR) imaging for early screening potentially provides faster and more accurate response; however, diagnosing COVID from CXRs is hard and we should rely on deep learning support, whose decision process is, on the other hand, "black-boxed" and, for such reason, untrustworthy. We propos…
BRAC University（BRAC大学）；Islamic University of Technology（伊斯兰技术大学）
In the world of medical diagnostics, the adoption of various deep learning techniques is quite common as well as effective, and its statement is equally true when it comes to implementing it into the retina Optical Coherence Tomography (OCT) sector, but (i)These techniques have the black box characteristics that prevent the medical professionals to completely trust the results generated from them (ii)Lack of precision of these methods restricts their implementation in clinical and complex cases (iii)The existing works and models on the OCT classification are substantially large and complicated…
Weakly-supervised segmentation using inherently-explainable classification models and their application to brain tumour classification
Knowledge Engineering Group, Faculty of Computer Science, Otto von Guericke University Magdeburg, Germany；Biomedical Magnetic Resonance, Faculty of Nature Sciences, Otto von Guericke University Magdeburg, Germany；Genomics Research Centre, Human Technopole, Milan, Italy；Institute for Medical Engineering, Faculty of Electrical Engineering；Information Technology, Otto von Guericke University Magdeburg, Germany；Digital Engineering Faculty, University of Potsdam, Germany；Machine Learning, Hasso-Plattner-Institute, Potsdam, Germany；Department of Biomedical Data Science, Stanford University, Stanford, CA, United States；Centre for Behavioural Brain Sciences, Magdeburg, Germany；German Centre for Neurodegenerative Disease, Magdeburg, Germany
Manchester Metropolitan University（曼彻斯特城市大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Computer-aided early diagnosis of Alzheimer's disease (AD) and its prodromal form mild cognitive impairment (MCI) based on structure Magnetic Resonance Imaging (sMRI) has provided a cost-effective and objective way for early prevention and treatment of disease progression, leading to improved patient care. In this work, we have proposed a novel computer-aided approach for early diagnosis of AD by introducing an explainable 3D Residual Attention Deep Neural Network (3D ResAttNet) for end-to-end learning from sMRI scans. Different from the existing approaches, the novelty of our approach is thre…
Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所IIS）；Nuremberg Institute of Technology Georg Simon Ohm（纽伦堡乔治·西蒙·欧姆技术学院）
Would you trust physicians if they cannot explain their decisions to you? Medical diagnostics using machine learning gained enormously in importance within the last decade. However, without further enhancements many state-of-the-art machine learning methods are not suitable for medical application. The most important reasons are insufficient data set quality and the black-box behavior of machine learning algorithms such as Deep Learning models. Consequently, end-users cannot correct the model's decisions and the corresponding explanations. The latter is crucial for the trustworthiness of machi…
Bridging visual saliency and large language models for explainable deep learning in medical imaging
Department of Mathematics and Computer science（数学与计算机科学系）；Institute of Cognitive Science（认知科学研究所）；Rhodes University（罗德斯大学）；National Institute for Theoretical and Computational Sciences (NITheCS)（理论与计算科学国家研究所）
Knee-xRAI: An Explainable AI Framework for Automatic Kellgren-Lawrence Grading of Knee Osteoarthritis
Orthopaedic Department, Faculty of Medicine UIN Syarif Hidayatullah Jakarta（乌姆尼大学医学学院骨科部）；Informatics Engineering, Institut Teknologi Sepuluh Nopember（十月份技术研究所信息工程系）；Information Technology, Universitas Muhammadiyah Yogyakarta（尤科阿卡塔大学信息技术系）；Industrial and Systems Engineering, King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学工业与系统工程系）
KAIST（韩国科学技术院）；Technical University of Munich（慕尼黑工业大学）
The ambiguity of the decision-making process has been pointed out as the main obstacle to applying the deep learning-based method in a practical way in spite of its outstanding performance. Interpretability could guarantee the confidence of deep learning system, therefore it is particularly important in the medical field. In this study, a novel deep network is proposed to explain the diagnostic decision with visual pointing map and diagnostic sentence justifying result simultaneously. For the purpose of increasing the accuracy of sentence generation, a visual word constraint model is devised i…
FunnyNodules: A Customizable Medical Dataset Tailored for Evaluating Explainable AI
Ulm University Medical Center（乌尔姆大学医学中心）；XAIRAD - Cooperation for Artificial Intelligence in Experimental Radiology（XAIRAD——实验放射学人工智能合作）；Department of Diagnostic and Interventional Radiology（诊断与介入放射学系）
MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering
Faculty Of Information Technology VNU University of Engineering（信息科技学院越南工程大学）；IT-BT Convergence Technology Division Vietnam-Korea Institute of Science（IT-BT融合技术部门越南-韩国科学技术院）；TADI Global Lab TADI Global Company Limited（TADI全球实验室TADI全球有限公司）；Faculty of Finance Banking Academy of Vietnam（金融学院越南银行学院）
OMNI-Dent: Towards an Accessible and Explainable AI Framework for Automated Dental Diagnosis
University of Minnesota（明尼苏达大学）；Khon Kaen University（科恩卡恩大学）；Minnesota State University（明尼苏达州立大学）；The University of Western Australia（西澳大学）；Kaohsiung Medical University（高雄医学院）；Auckland University of Technology（奥克兰理工大学）
Dermatologist-like explainable AI enhances melanoma diagnosis accuracy: eye-tracking study
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Medical University of Vienna（维也纳医科大学）；Uniklinikum Erlangen（埃尔朗根大学医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大-埃尔朗根-纽伦堡大学）；Bavarian Cancer Research Center (BZFK)（巴伐利亚癌症研究中心）；University Medical Center Mannheim（曼海姆大学医学中心）；Ruprecht-Karl University of Heidelberg（海德堡鲁普雷希特-卡尔大学（海德堡大学））；DKFZ Hector Cancer Institute at the University Medical Center Mannheim（曼海姆大学医学中心DKFZ赫克托癌症研究所）；University Hospital Frankfurt（法兰克福大学医院）；Goethe-University Frankfurt（法兰克福歌德大学）；Pontificia Universidad Católica de Chile（智利天主教大学）；University Hospital Würzburg（维尔茨堡大学医院）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；TUD Dresden University of Technology（德累斯顿工业大学）；University Hospital Aachen（亚琛大学医院）；RWTH University of Aachen（亚琛工业大学）
Minimum Data, Maximum Impact: 20 annotated samples for explainable lung nodule classification
Experimental Radiology, Ulm University Medical Center, Ulm, Germany（实验放射学，乌尔姆大学医学中心，乌尔姆，德国）；Department of Diagnostic and Interventional Radiology, Ulm University Medical Center, Ulm, Germany（诊断与介入放射学系，乌尔姆大学医学中心，乌尔姆，德国）；XAIRAD - Cooperation for Artificial Intelligence in Experimental Radiology, Ulm, Germany（XAIRAD - 实验放射学中的人工智能合作）
Weed Detection in Challenging Field Conditions: A Semi-Supervised Framework for Overcoming Shadow Bias and Data Scarcity
College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；Agriculture Technology and Adoption Centre, James Cook University（詹姆斯库克大学农业技术与采纳中心）；ARC Training Centre in Plant Biosecurity（植物生物安全培训中心）
Hosei University（法政大学）；Tokyo Women’s Medical University（东京女子医科大学）
Although highly accurate automated diagnostic techniques for melanoma have been reported, the realization of a system capable of providing diagnostic evidence based on medical indices remains an open issue because of difficulties in obtaining reliable training data. In this paper, we propose bulk production augmentation (BPA) to generate high-quality, diverse pseudo-skin tumor images with the desired structural malignant features for additional training images from a limited number of labeled images. The proposed BPA acts as an effective data augmentation in constructing the feature detector f…
Crime and Security Research Institute（犯罪与安全研究所）；Cardiff University（卡迪夫大学）；School of Engineering（工程学院）
The popularity of Deep Learning for real-world applications is ever-growing. With the introduction of high performance hardware, applications are no longer limited to image recognition. With the introduction of more complex problems comes more and more complex solutions, and the increasing need for explainable AI. Deep Neural Networks for Video tasks are amongst the most complex models, with at least twice the parameters of their Image counterparts. However, explanations for these models are often ill-adapted to the video domain. The current work in explainability for video models is still ove…
A Deep Learning Approach for Automated Skin Lesion Diagnosis with Explainable AI
1 ,2,3Department of Computer Science \& Engineering, Green University of Bangladesh, Dhaka, Bangladesh；4Department of Computer Science \& Engineering, Bangladesh University of Business；5Department of Internet of Things \& Robotics Engineering, University of Frontier Technology, Bangladesh
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Royal Brompton Hospital（皇家布朗普顿医院）；Shenzhen Second People’s Hospital（深圳市第二人民医院）；TECNALIA（泰克纳利亚研究中心）；University of the Basque Country（巴斯克大学）
The upheaval brought by the arrival of the COVID-19 pandemic has continued to bring fresh challenges over the past two years. During this COVID-19 pandemic, there has been a need for rapid identification of infected patients and specific delineation of infection areas in computed tomography (CT) images. Although deep supervised learning methods have been established quickly, the scarcity of both image-level and pixel-level labels as well as the lack of explainable transparency still hinder the applicability of AI. Can we identify infected patients and delineate the infections with extreme mini…
A Graph-Augmented knowledge Distillation based Dual-Stream Vision Transformer with Region-Aware Attention for Gastrointestinal Disease Classification with Explainable AI
Department of Computer Science and Engineering（计算机科学与工程系）
ProjectedEx: Enhancing Generation in Explainable AI for Prostate Cancer
Flinders University（弗林德斯大学）；The University of Adelaide（阿德莱德大学）；The Australian National University（澳大利亚国立大学）；The University of Sydney（悉尼大学）；Zhejiang University of Technology（浙江工业大学）；Guangdong Technion – Israel Institute of Technology（广东以色列理工学院）；University of Chinese Academy of Sciences（中国科学院大学）；Rice University（莱斯大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；La Trobe University（拉筹伯大学）
