Explainable Artificial Intelligence (AI) in the form of an interpretable and semiautomatic approach to stage grading ocular pathologies such as Diabetic retinopathy, Hypertensive retinopathy, and other retinopathies on the backdrop of major systemic diseases. The experimental study aims to evaluate an explainable staged grading process without using deep Convolutional Neural Networks (CNNs) directly. Many current CNN-based deep neural networks used for diagnosing retinal disorders might have appreciable performance but fail to pinpoint the basis driving their decisions. To improve these decisions' transparency, we have proposed a clinician-in-the-loop assisted intelligent workflow that performs a retinal vascular assessment on the fundus images to derive quantifiable and descriptive parameters. The retinal vessel parameters meta-data serve as hyper-parameters for better interpretation and explainability of decisions. The semiautomatic methodology aims to have a federated approach to AI in healthcare applications with more inputs and interpretations from clinicians. The baseline process involved in the machine learning pipeline through image processing techniques for optic disc detection, vessel segmentation, and arteriole/venule identification.
OCTCube-M: A 3D multimodal optical coherence tomography foundation model for retinal and systemic diseases with cross-cohort and cross-device validation
An Explainable Transformer Model for Alzheimer's Disease Detection Using Retinal Imaging
Nanotechnology, Biotechnology, Information Technology and Cognitive Science Laboratory, University of Tehran（特拉布宗大学纳米技术、生物技术、信息技术和认知科学实验室）；Department of Mechatronics, School of Intelligent Systems, College of Interdisciplinary Science and Technology, University of Tehran（特拉布宗大学机电系，智能系统学院，跨学科科学与技术学院）；School of Science and Technology, Faculty of Science, Agriculture, Business and Law, University of New England（新英格兰大学科学与技术学院，科学、农业、商业与法律学院）；Graduate School of Health, University of Technology Sydney（悉尼技术大学健康研究生院）；School of Medicine and Public Health, College of Health, Medicine and Wellbeing, The University of Newcastle（新castle大学医学与公共卫生学院，健康、医学与福祉学院）
BRAC University（BRAC大学）；Islamic University of Technology（伊斯兰技术大学）
In the world of medical diagnostics, the adoption of various deep learning techniques is quite common as well as effective, and its statement is equally true when it comes to implementing it into the retina Optical Coherence Tomography (OCT) sector, but (i)These techniques have the black box characteristics that prevent the medical professionals to completely trust the results generated from them (ii)Lack of precision of these methods restricts their implementation in clinical and complex cases (iii)The existing works and models on the OCT classification are substantially large and complicated…
Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images
Ahsanullah University of Science and Technology（阿萨努拉科技大学）
The Pingry School（平格里学校）；Chinle Comprehensive Health Care Facility（钦利综合医疗中心）；University of Maryland, College Park（马里兰大学帕克分校）；NVIDIA（英伟达）
30 million Optical Coherence Tomography (OCT) imaging tests are issued annually to diagnose various retinal diseases, but accurate diagnosis of OCT scans requires trained eye care professionals who are still prone to making errors. With better systems for diagnosis, many cases of vision loss caused by retinal disease could be entirely avoided. In this work, we present ReLaX, a novel deep learning framework for explainable, accurate classification of retinal pathologies which achieves state-of-the-art accuracy. Furthermore, we emphasize producing both qualitative and quantitative explanations o…
The Role of AI in Early Detection of Life-Threatening Diseases: A Retinal Imaging Perspective
Naif Arab University for Security Sciences（奈伊夫阿拉伯安全科学大学）；Charles Sturt University（查尔斯特大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
University of Waterloo（滑铁卢大学）；Medical Research Foundation（医学研究基金会）；Sankara Nethralaya（桑卡拉·内特拉拉亚眼科医院）
Background: The lack of explanations for the decisions made by algorithms such as deep learning has hampered their acceptance by the clinical community despite highly accurate results on multiple problems. Recently, attribution methods have emerged for explaining deep learning models, and they have been tested on medical imaging problems. The performance of attribution methods is compared on standard machine learning datasets and not on medical images. In this study, we perform a comparative analysis to determine the most suitable explainability method for retinal OCT diagnosis. Methods: A com…
The status of retinal arteriovenous crossing is of great significance for clinical evaluation of arteriolosclerosis and systemic hypertension. As an ophthalmology diagnostic criteria, Scheie's classification has been used to grade the severity of arteriolosclerosis. In this paper, we propose a deep learning approach to support the diagnosis process, which, to the best of our knowledge, is one of the earliest attempts in medical imaging. The proposed pipeline is three-fold. First, we adopt segmentation and classification models to automatically obtain vessels in a retinal image with the corresp…
Machine Learning Prediction of Cardiovascular Risk in Type 1 Diabetes Mellitus Using Radiomics Features from Multimodal Retinal Images
Universitat Politècnica de Catalunya (UPC)（加泰罗尼亚理工大学）；Institut Clínic d´Oftalmología (ICOF)（眼科临床研究所）；Hospital Clínic de Barcelona（巴塞罗那临床医院）；Intelligent Data Science and Artificial Intelligence (IDEAI-UPC) Research Center（智能数据科学与人工智能研究中心）；August Pi i Sunyer Biomedical Research Institute (IDIBAPS)（奥古斯特·皮伊·苏涅尔生物医学研究所）；School of Medicine, Universitat de Barcelona（巴塞罗那大学医学院）
JIS College of Engineering（JIS工程学院）；Kalyani Govt. Engineering College（卡利亚尼政府工程学院）
Segmentation of blood vessels in retinal images provides early diagnosis of diseases like glaucoma, diabetic retinopathy and macular degeneration. Among these diseases occurrence of Glaucoma is most frequent and has serious ocular consequences that can even lead to blindness, if it is not detected early. The clinical criteria for the diagnosis of glaucoma include intraocular pressure measurement, optic nerve head evaluation, retinal nerve fiber layer and visual field defects. This form of blood vessel segmentation helps in early detection for ophthalmic diseases, and potentially reduces the ri…
Explainable Multi-Task Retinal Imaging Reveals Microvascular Signals for Systemic Risk Stratification in Type 2 Diabetes: A Pilot Study
Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology（深圳先进技术大学计算机科学与人工智能学院）；Frontier Science Computing Center, Zhuhai Institute of Advanced Technology Chinese Academy of Sciences（中国科学院珠海先进技术研究院前沿科学计算中心）；Chinese University of Hong Kong（香港中文大学）；Zhuhai People's Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University)（珠海人民医院（北京理工大学珠海临床医学院附属医院））；Lampang Inter-Tech College, Lampang Thailand（泰国 Lampang 职业技术学院）
Prediction of Alzheimer's Disease Risk Factors from Retinal Images via Deep Learning: Development and Validation of Biologically Relevant Morphological Associations in the UK Biobank
J. Crayton Pruitt Family Dept. of Biomedical Engineering, University of Florida（朱·克雷顿·普瑞特生物医学工程系，佛罗里达大学）；University of Florida Research Computing（佛罗里达大学研究计算中心）；Meta AI (FAIR)（Meta AI（FAIR））；School of Behavioral and Brain Sciences, University of Texas at Dallas（德克萨斯大学达拉斯分校行为与脑科学学院）；Dept. of Electrical and Computer Engineering, University of Florida（佛罗里达大学电气与计算机工程系）；Dept. of Computer and Information Science and Engineering, University of Florida（佛罗里达大学计算机与信息科学与工程系）；Center for Cognitive Aging and Memory, University of Florida（佛罗里达大学认知衰老与记忆中心）
Purpose: Convolutional neural networks can be trained to detect various conditions or patient traits based on retinal fundus photographs, some of which, such as the patient sex, are invisible to the expert human eye. Here we propose a methodology for explainable classification of fundus images to uncover the mechanism(s) by which CNNs successfully predict the labels. We used patient sex as a case study to validate our proposed methodology. Approach: First, we used a set of 4746 fundus images, including training, validation and test partitions, to fine-tune a pre-trained CNN on the sex classifi…
A Disease-Specific Foundation Model Using Over 100K Fundus Images: Release and Validation for Abnormality and Multi-Disease Classification on Downstream Tasks
Seoul National University（首尔大学）；Seoul National University Hospital（首尔大学附属医院）
A Hybrid Fully Convolutional CNN-Transformer Model for Inherently Interpretable Disease Detection from Retinal Fundus Images
Hertie Institute for AI in Brain Health, University of Tübingen, Germany（人工智能与脑健康赫尔特研究所，图宾根大学，德国）；Tübingen AI Center, University of Tübingen, Germany（图宾根人工智能中心，图宾根大学，德国）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Stein Eye Institute（斯坦眼科研究所）；VoxelCloud, Inc.（体素云公司）
The retinal vasculature provides important clues in the diagnosis and monitoring of systemic diseases including hypertension and diabetes. The microvascular system is of primary involvement in such conditions, and the retina is the only anatomical site where the microvasculature can be directly observed. The objective assessment of retinal vessels has long been considered a surrogate biomarker for systemic vascular diseases, and with recent advancements in retinal imaging and computer vision technologies, this topic has become the subject of renewed attention. In this paper, we present a novel…
Deep learning methods for ophthalmic diagnosis have shown considerable success in tasks like segmentation and classification. However, their widespread application is limited due to the models being opaque and vulnerable to making a wrong decision in complicated cases. Explainability methods show the features that a system used to make prediction while uncertainty awareness is the ability of a system to highlight when it is not sure about the decision. This is one of the first studies using uncertainty and explanations for informed clinical decision making. We perform uncertainty analysis of a…
Beihang University（北京航空航天大学）；Peng Cheng Laboratory（鹏城实验室）；RIKEN AIP（理化学研究所人工智能研究中心）；University of Tokyo（东京大学）；Ningbo Institute of Industrial Technology, CAS（中国科学院宁波工业技术研究院）
Though deep learning has shown successful performance in classifying the label and severity stage of certain diseases, most of them give few explanations on how to make predictions. Inspired by Koch's Postulates, the foundation in evidence-based medicine (EBM) to identify the pathogen, we propose to exploit the interpretability of deep learning application in medical diagnosis. By determining and isolating the neuron activation patterns on which diabetic retinopathy (DR) detector relies to make decisions, we demonstrate the direct relation between the isolated neuron activation and lesions for…
Universidade da Coruña（拉科鲁尼亚大学）；Instituto de Investigación Biomédica de A Coruña (INIBIC)（拉科鲁尼亚生物医学研究所）；Hospital Clínico San Carlos（圣卡洛斯临床医院）；Instituto de Investigación Sanitaria (IdISSC)（圣卡洛斯健康研究所）；Centro Internacional de Oftalmología Avanzada（国际高级眼科中心）
Age-related macular degeneration (AMD) is a degenerative disorder affecting the macula, a key area of the retina for visual acuity. Nowadays, it is the most frequent cause of blindness in developed countries. Although some promising treatments have been developed, their effectiveness is low in advanced stages. This emphasizes the importance of large-scale screening programs. Nevertheless, implementing such programs for AMD is usually unfeasible, since the population at risk is large and the diagnosis is challenging. All this motivates the development of automatic methods. In this sense, severa…
Pulsatile changes in retinal vascular geometry over the cardiac cycle have clinical implications for the diagnosis of ocular and systemic vascular diseases, including ischemia, coronary heart diseases, and diabetes mellitus and its complications. Thus, analysis of the pulsatile changes over the cardiac cycle is a potential non-invasive assessment for the presence of ocular and systemic vascular diseases. The cardiac rhythm influences these pulsatile changes in the retina, is a result of the change in blood volumetric flow entering the ophthalmic-vascular system under a certain level of intraoc…
TWLR: Text-Guided Weakly-Supervised Lesion Localization and Severity Regression for Explainable Diabetic Retinopathy Grading
Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science（广东省级交叉学科研究与数据科学应用重点实验室）；Department of Statistics and Data Science, Beijing Normal-Hong Kong Baptist University（北京师范大学-香港 Baptist大学统计与数据科学系）；Faculty of Science, Hong Kong Baptist University（香港 Baptist大学科学学院）；Data Science Research Center, Duke Kunshan University（杜克-昆山大学数据科学研究中心）；Shanxi Provincial People’s Hospital（山西人民医院）；The Fifth Clinical Medical school of Shanxi Medical University（山西医科大学第五临床医学院）；Research Center for Mathematics, Beijing Normal University（北京师范大学数学研究中心）；Department of Mathematics and Statistics, York University（约克大学数学与统计学系）
The accurate retinal vessel segmentation (RVS) is of great significance to assist doctors in the diagnosis of ophthalmology diseases and other systemic diseases. Manually designing a valid neural network architecture for retinal vessel segmentation requires high expertise and a large workload. In order to improve the performance of vessel segmentation and reduce the workload of manually designing neural network, we propose novel approach which applies neural architecture search (NAS) to optimize an encoder-decoder architecture for retinal vessel segmentation. A modified evolutionary algorithm …
Fine-tuning Vision Language Models with Graph-based Knowledge for Explainable Medical Image Analysis
School of Electrical and Computer Engineering, Cornell University（电气与计算机工程学院，康奈尔大学）；School of Computation, Information and Technology, Technical University of Munich（计算、信息与技术学院，慕尼黑技术大学）；Cornell Tech（康奈尔科技）；Weill Cornell Medicine（韦尔·康奈尔医学）
Systemic diseases such as diabetes, hypertension, atherosclerosis are among the leading causes of annual human mortality rate. It is suggested that retinal and conjunctival vascular tortuosity is a potential biomarker for such systemic diseases. Most importantly, it is observed that the tortuosity depends on the thickness of these vessels. Therefore, selective calculation of tortuosity within specific vessel thicknesses is required depending on the disease being analysed. In this paper, we propose a thickness sensitive vessel extraction framework that is primarily applicable for studies relate…
Alzheimer's Disease (AD) is a progressive neurodegenerative disease and the leading cause of dementia. Early diagnosis is critical for patients to benefit from potential intervention and treatment. The retina has been hypothesized as a diagnostic site for AD detection owing to its anatomical connection with the brain. Developed AI models for this purpose have yet to provide a rational explanation about the decision and neither infer the stage of disease's progression. Along this direction, we propose a novel model-agnostic explainable-AI framework, called Granular Neuron-level Explainer (LAVA)…
University of Rostock（罗斯托克大学）；Rostock University Medical Center（罗斯托克大学医学中心）
Advances in optical coherence tomography (OCT) have enabled noninvasive imaging of substructures of the human retina with high spatial resolution. OCT examinations are now a standard procedure in clinics and an integral part of ophthalmic research. The interpretation of the OCT helps ophthalmologists understand the impact of various retinal and systemic diseases on the structure of the retina in a way not previously possible. In the early stages of retinal diseases, however, the identification and analysis of small and localized substructural changes in the retina remains a challenge. We prese…
A General Bézier Tree Encoding Counterfactual Framework for Retinal-Vessel-Mediated Disease Analysis
Department of Electronic and Electrical Engineering（电子与电气工程系）；School of Data Science（数据科学学院）；Research Institute of Electrical Communication（电气通信研究所）；J. Crayton Pruitt Family Department of Biomedical Engineering（姜·克雷顿·普瑞特家庭生物医学工程系）
FusionFM: Fusing Eye-specific Foundational Models for Optimized Ophthalmic Diagnosis
Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore（新加坡国立大学 Yong Loo Lin 医学院眼科系）；Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore, Singapore（新加坡国立大学 Yong Loo Lin 医学院创新与精准眼科健康中心）；Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore（新加坡眼科研究院）；Ophthalmology and Visual Science Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore（杜克-国立新加坡大学医学学校眼科与视觉科学学术临床项目）；Centre for Medical Image Computing, University College London, London, United Kingdom（伦敦大学学院医学图像计算中心）；Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, Shantou, China（汕尾国际眼科中心，汕尾大学及香港中文大学）；Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland（苏黎世州立医院眼科部）；Spross Research Institute, Zurich, Switzerland（苏黎世 Spross 研究院）；Institute of High Performance Computing, A*STAR, Singapore（新加坡 A*STAR 高性能计算研究院）；NIHR Biomedical Research Centre, Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom（英国国家健康服务基金会 Moorfields 眼科医院生物医学研究中心）；Institute of Ophthalmology, University College London, London, United Kingdom（伦敦大学学院眼科研究院）
FairDomain: Achieving Fairness in Cross-Domain Medical Image Segmentation and Classification
Center for Artificial Intelligence and Robotics, New York University Abu Dhabi（纽约大学阿布扎比分校人工智能与机器人中心）；New York University（纽约大学）；Harvard University（哈佛大学）
Artificial intelligence techniques in inherited retinal diseases: A review
The University of Western Australia（西澳大利亚大学）；School of Allied Health（联合健康学院）；School of Physics, Maths and Computing（物理、数学与计算学院）；Centre for Ophthalmology and Visual Sciences (Lions Eye Institute)（眼科与视觉科学中心（狮子眼科研究所））
University of Science and Technology of China（中国科学技术大学）；The Ohio State University（俄亥俄州立大学）
Early and accurate diagnosis of Alzheimer's disease (AD) and its prodromal period mild cognitive impairment (MCI) is essential for the delayed disease progression and the improved quality of patients'life. The emerging computer-aided diagnostic methods that combine deep learning with structural magnetic resonance imaging (sMRI) have achieved encouraging results, but some of them are limit of issues such as data leakage and unexplainable diagnosis. In this research, we propose a novel end-to-end deep learning approach for automated diagnosis of AD and localization of important brain regions rel…
Generalist versus Specialist Vision Foundation Models for Ocular Disease and Oculomics
Institute of Ophthalmology, University College London（伦敦大学学院眼科研究所）；Hawkes Institute, University College London（伦敦大学学院霍克斯研究所）；NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust（莫菲尔德眼科医院 NHS 基金会信托 NIHR 生物医学研究中心）；Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院）；Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院创新与精准眼健康中心）；Singapore Eye Research Institute, Singapore National Eye Centre（新加坡国家眼科中心新加坡眼科研究所）；Chinese University of Hong Kong（香港中文大学）；Shanghai Jiao Tong University（上海交通大学）；MOE Key Laboratory of AI, School of Electronic, Information, and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院教育部人工智能重点实验室）；Stadtspital Zürich（苏黎世市立医院）；Spross Research Institute（斯普罗斯研究所）；Tsinghua Medicine, Tsinghua University（清华大学清华医学）；Beijing Visual Science and Translational Eye Research Institute, Beijing Tsinghua Changgung Hospital（北京清华长庚医院北京视觉科学与转化眼科研究所）；University College London（伦敦大学学院）
Background: Several studies indicate that the anterior visual pathway provides information about the dynamics of axonal degeneration in Multiple Sclerosis (MS). Current research in the field is focused on the quest for the most discriminative features among patients and controls and the development of machine learning models that yield computer-aided solutions widely usable in clinical practice. However, most studies are conducted with small samples and the models are used as black boxes. Clinicians should not trust machine learning decisions unless they come with comprehensive and easily unde…
Is an Ultra Large Natural Image-Based Foundation Model Superior to a Retina-Specific Model for Detecting Ocular and Systemic Diseases?
National University of Singapore（新加坡国立大学）；Northeastern University（东北大学）；University College London（伦敦大学学院）；NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust（莫菲尔德眼科医院 NHS 基金会信托 NIHR 生物医学研究中心）；Singapore Eye Research Institute, Singapore National Eye Centre（新加坡眼科研究所，新加坡国家眼科中心）；Institute of High-Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）；The Chinese University of Hong Kong（香港中文大学）；Stadtspital Zürich（苏黎世市立医院）；Spross Research Institute（斯普罗斯研究所）；Semmelweis University（塞梅尔维斯大学）；Shantou University（汕头大学）；Shantou University Medical College（汕头大学医学院）
