We analyze a dataset of retinal images using linear probes: linear regression models trained on some "target" task, using embeddings from a deep convolutional (CNN) model trained on some "source" task as input. We use this method across all possible pairings of 93 tasks in the UK Biobank dataset of retinal images, leading to ~164k different models. We analyze the performance of these linear probes by source and target task and by layer depth. We observe that representations from the middle layers of the network are more generalizable. We find that some target tasks are easily predicted irrespective of the source task, and that some other target tasks are more accurately predicted from correlated source tasks than from embeddings trained on the same task.
Generative artificial intelligence in ophthalmology: multimodal retinal images for the diagnosis of Alzheimer's disease with convolutional neural networks
Explainable Artificial Intelligence (AI) in the form of an interpretable and semiautomatic approach to stage grading ocular pathologies such as Diabetic retinopathy, Hypertensive retinopathy, and other retinopathies on the backdrop of major systemic diseases. The experimental study aims to evaluate an explainable staged grading process without using deep Convolutional Neural Networks (CNNs) directly. Many current CNN-based deep neural networks used for diagnosing retinal disorders might have appreciable performance but fail to pinpoint the basis driving their decisions. To improve these decisi…
Casey Eye Institute, Oregon Health & Science University（俄勒冈健康与科学大学凯西眼科研究所）；School of Physics and Optoelectronic Engineering, Foshan University（佛山大学物理与光电工程学院）；Department of Biomedical Engineering, Oregon Health & Science University（俄勒冈健康与科学大学生物医学工程系）
Purpose: We proposed a deep convolutional neural network (CNN), named Retinal Fluid Segmentation Network (ReF-Net) to segment volumetric retinal fluid on optical coherence tomography (OCT) volume. Methods: 3 x 3-mm OCT scans were acquired on one eye by a 70-kHz OCT commercial AngioVue system (RTVue-XR; Optovue, Inc.) from 51 participants in a clinical diabetic retinopathy (DR) study (45 with retinal edema and 6 healthy controls). A CNN with U-Net-like architecture was constructed to detect and segment the retinal fluid. Cross-sectional OCT and angiography (OCTA) scans were used for training an…
A Hybrid Fully Convolutional CNN-Transformer Model for Inherently Interpretable Disease Detection from Retinal Fundus Images
Hertie Institute for AI in Brain Health, University of Tübingen, Germany（人工智能与脑健康赫尔特研究所，图宾根大学，德国）；Tübingen AI Center, University of Tübingen, Germany（图宾根人工智能中心，图宾根大学，德国）
A BERT-Style Self-Supervised Learning CNN for Disease Identification from Retinal Images
School of Computing and Augmented Intelligence, Arizona State University（计算与增强智能学院，亚利桑那州立大学）；McKeley School of Engineering, Washington University in St. Louis（工程学院，华盛顿大学圣路易斯分校）；Department of Neurology, Mayo Clinic（神经病学部，梅奥诊所）
Shanghai Jiao Tong University（上海交通大学）；EPFL（洛桑联邦理工学院）；Shanghai General Hospital（上海市第一人民医院）
Projection map (PM) from optical coherence tomography (OCT) B-scan is an important tool to diagnose retinal diseases, which typically requires retinal layer segmentation. In this study, we present a novel end-to-end framework to predict PMs from B-scans. Instead of segmenting retinal layers explicitly, we represent them implicitly as predicted coordinates. By pixel interpolation on uniformly sampled coordinates between retinal layers, the corresponding PMs could be easily obtained with pooling. Notably, all the operators are differentiable; therefore, this Differentiable Projection Module (DPM…
Oregon Health & Science University（俄勒冈健康与科学大学）；MGH/Harvard Medical School（麻省总医院/哈佛医学院）；MGH & BWH Center for Clinical Data Science（麻省总医院与布莱根妇女医院临床数据科学中心）；University of Lincoln（林肯大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；National Eye Institute, National Institutes of Health（美国国立卫生研究院国家眼科研究所）
Background: Artificial intelligence (AI) may demonstrate racial bias when skin or choroidal pigmentation is present in medical images. Recent studies have shown that convolutional neural networks (CNNs) can predict race from images that were not previously thought to contain race-specific features. We evaluate whether grayscale retinal vessel maps (RVMs) of patients screened for retinopathy of prematurity (ROP) contain race-specific features. Methods: 4095 retinal fundus images (RFIs) were collected from 245 Black and White infants. A U-Net generated RVMs from RFIs, which were subsequently thr…
Segmentation of Retinal Low-Cost Optical Coherence Tomography Images using Deep Learning
Institute of Medical Informatics, University of Lübeck（吕贝克大学医学信息学研究所）；University of Lübeck（吕贝克大学）；Graduate School for Computing in Medicine and Life Sciences, University of Lübeck（医学与生命科学计算研究生学院，吕贝克大学）；Medical Laser Center Lübeck（吕贝克医疗激光中心）；Department of Ophthalmology, University of Kiel（基尔大学眼科部门）；Institute of Biomedical Optics, University of Lübeck（吕贝克大学生物医学光学研究所）
Purpose: Convolutional neural networks can be trained to detect various conditions or patient traits based on retinal fundus photographs, some of which, such as the patient sex, are invisible to the expert human eye. Here we propose a methodology for explainable classification of fundus images to uncover the mechanism(s) by which CNNs successfully predict the labels. We used patient sex as a case study to validate our proposed methodology. Approach: First, we used a set of 4746 fundus images, including training, validation and test partitions, to fine-tune a pre-trained CNN on the sex classifi…
Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images
Ahsanullah University of Science and Technology（阿萨努拉科技大学）
The Pingry School（平格里学校）；Chinle Comprehensive Health Care Facility（钦利综合医疗中心）；University of Maryland, College Park（马里兰大学帕克分校）；NVIDIA（英伟达）
30 million Optical Coherence Tomography (OCT) imaging tests are issued annually to diagnose various retinal diseases, but accurate diagnosis of OCT scans requires trained eye care professionals who are still prone to making errors. With better systems for diagnosis, many cases of vision loss caused by retinal disease could be entirely avoided. In this work, we present ReLaX, a novel deep learning framework for explainable, accurate classification of retinal pathologies which achieves state-of-the-art accuracy. Furthermore, we emphasize producing both qualitative and quantitative explanations o…
Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retinal OCT Images with Full and Sparse Annotations
Xiamen University（厦门大学）；National Institute for Data Science in Health and Medicine, Xiamen University（厦门大学健康医疗大数据国家研究院）；Tencent YouTu Lab（腾讯优图实验室）；Southern University of Science and Technology（南方科技大学）
University of Tehran（德黑兰大学）；Noor Ophthalmology Research Center, Noor Eye Hospital（努尔眼科研究中心，努尔眼科医院）；Henry Ford Health System（亨利·福特医疗系统）
Age-related macular degeneration (AMD) is the most common cause of blindness in developed countries, especially in people over 60 years of age. The workload of specialists and the healthcare system in this field has increased in recent years mainly due to the prevalence of population aging worldwide and the chronic nature of AMD. Recent developments in deep learning have provided a unique opportunity to develop fully automated diagnosis frameworks. Considering the presence of AMD-related retinal pathologies in varying sizes in OCT images, our objective was to propose a multi-scale convolutiona…
Jadavpur University（贾达布尔大学）；IIT Madras（印度理工学院马德拉斯分校）；IIT Jodhpur（印度理工学院焦特布尔分校）；Inception Institute of AI（Inception人工智能研究院）；Monash University（莫纳什大学）；Airdoc-Monash Research（Airdoc-莫纳什研究中心）
Convolutional Neural Network models have successfully detected retinal illness from optical coherence tomography (OCT) and fundus images. These CNN models frequently rely on vast amounts of labeled data for training, difficult to obtain, especially for rare diseases. Furthermore, a deep learning system trained on a data set with only one or a few diseases cannot detect other diseases, limiting the system's practical use in disease identification. We have introduced an unsupervised approach for detecting anomalies in retinal images to overcome this issue. We have proposed a simple, memory effic…
Divyadrishti Imaging Laboratory, IIT Roorkee, Indian Institute of Technology Roorkee（IIT Roorkee 的 Divyadrishti 成像实验室，印度理工学院 Roorkee）
Image data set from a multi-spectral animal imaging system is used to address two issues: (a) registering the oscillation in optical coherence tomography (OCT) images due to mouse eye movement and (b) suppressing the shadow region under the thick vessels/structures. Several classical and AI-based algorithms in combination are tested for each task to see their compatibility with data from the combined animal imaging system. Hybridization of AI with optical flow followed by Homography transformation is shown to be working (correlation value>0.7) for registration. Resnet50 backbone is shown to be…
With the advancement of powerful image processing and machine learning techniques, CAD has become ever more prevalent in all fields of medicine including ophthalmology. Since optic disc is the most important part of retinal fundus image for glaucoma detection, this paper proposes a two-stage framework that first detects and localizes optic disc and then classifies it into healthy or glaucomatous. The first stage is based on RCNN and is responsible for localizing and extracting optic disc from a retinal fundus image while the second stage uses Deep CNN to classify the extracted disc into health…
OCTDL: Optical Coherence Tomography Dataset for Image-Based Deep Learning Methods
Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大-埃尔朗根-纽伦堡大学）；Ural Federal University Named after the First President of Russia B. N. Yeltsin（俄罗斯首任总统B.N.叶利钦命名的乌拉尔联邦大学）；Ophthalmosurgery Clinic "Professorskaya Plus"（“Professorskaya Plus”眼科诊所）；Ural State Medical University（乌拉尔国立医科大学）
Interpretable Retinal Disease Prediction Using Biology-Informed Heterogeneous Graph Representations
Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；School of Medicine and Health, TUM University Hospital（医学与健康学院，塔尔博恩大学医院）；School of Computation, Information and Technology, TUM（计算、信息与技术学院，塔尔博恩大学）；Weill Cornell Medicine, Cornell University, New York City（韦尔·科恩医学中心，康奈尔大学，纽约市）
BRAC University（BRAC大学）；Islamic University of Technology（伊斯兰技术大学）
In the world of medical diagnostics, the adoption of various deep learning techniques is quite common as well as effective, and its statement is equally true when it comes to implementing it into the retina Optical Coherence Tomography (OCT) sector, but (i)These techniques have the black box characteristics that prevent the medical professionals to completely trust the results generated from them (ii)Lack of precision of these methods restricts their implementation in clinical and complex cases (iii)The existing works and models on the OCT classification are substantially large and complicated…
Automated surface segmentation of retinal layer is important and challenging in analyzing optical coherence tomography (OCT). Recently, many deep learning based methods have been developed for this task and yield remarkable performance. However, due to large spatial gap and potential mismatch between the B-scans of OCT data, all of them are based on 2D segmentation of individual B-scans, which may loss the continuity information across the B-scans. In addition, 3D surface of the retina layers can provide more diagnostic information, which is crucial in quantitative image analysis. In this stud…
PiPViT: Patch-based Visual Interpretable Prototypes for Retinal Image Analysis
Christian Doppler Laboratory for Artificial Intelligence in Retina（人工智能在视网膜领域的克里斯蒂安·多普勒实验室）；Institute of Artificial Intelligence（人工智能研究所）；Center for Medical Data Science（医学数据科学中心）；Medical University of Vienna（维也纳医学大学）
Optical Coherence Tomography is a technique used to scan the Retina of the eye and check for tears. In this paper, we develop a Convolutional Neural Network Architecture for OCT scan classification. The model is trained to detect Retinal tears from an OCT scan and classify the type of tear. We designed a block-based approach to accompany a pre-trained VGG-19 using Transfer Learning by writing customised layers in blocks for better feature extraction. The approach achieved substantially better results than the baseline we initially started out with.
Universidad Nacional de Colombia（哥伦比亚国立大学）；Fundación Oftalmológica Nacional（国家眼科基金会）；Universidad del Rosario（罗萨里奥大学）
Diabetic retinopathy (DR) is the result of a complication of diabetes affecting the retina. It can cause blindness, if left undiagnosed and untreated. An ophthalmologist performs the diagnosis by screening each patient and analyzing the retinal lesions via ocular imaging. In practice, such analysis is time-consuming and cumbersome to perform. This paper presents a model for automatic DR classification on eye fundus images. The approach identifies the main ocular lesions related to DR and subsequently diagnoses the illness. The proposed method follows the same workflow as the clinicians, provid…
Carnegie Mellon University（卡内基梅隆大学）；INESC TEC；Faculty of Engineering, University of Porto（波尔图大学工程学院）
We propose a pixel color amplification theory and family of enhancement methods to facilitate segmentation tasks on retinal images. Our novel re-interpretation of the image distortion model underlying dehazing theory shows how three existing priors commonly used by the dehazing community and a novel fourth prior are related. We utilize the theory to develop a family of enhancement methods for retinal images, including novel methods for whole image brightening and darkening. We show a novel derivation of the Unsharp Masking algorithm. We evaluate the enhancement methods as a pre-processing step…
Pretrained Deep 2.5D Models for Efficient Predictive Modeling from Retinal OCT
University of Basel（巴塞尔大学）；Technical University Munich（慕尼黑工业大学）；Medical University of Vienna（维也纳医科大学）；Moorfields Eye Hospital NHS Foundation Trust（摩尔菲尔斯眼科医院NHS基金会信托）；Imperial College London（帝国理工学院）；Institute of Molecular and Clinical Ophthalmology Basel（巴塞尔分子与临床眼科研究所）；University of Southampton（南安普顿大学）
The Hong Kong University of Science and Technology（香港科技大学）
Deep learning techniques have shown their success in medical image segmentation since they are easy to manipulate and robust to various types of datasets. The commonly used loss functions in the deep segmentation task are pixel-wise loss functions. This results in a bottleneck for these models to achieve high precision for complicated structures in biomedical images. For example, the predicted small blood vessels in retinal images are often disconnected or even missed under the supervision of the pixel-wise losses. This paper addresses this problem by introducing a long-range elastic interacti…
Discriminating retinal microvascular and neuronal differences related to migraines: Deep Learning based Crossectional Study
Monash University（莫纳什大学）；University of New South Wales（新南威尔士大学）；School of Optometry & Vision Science（视光与视觉科学学院）；School of Computer Science and Engineering（计算机科学与工程学院）
Detection and Classification of Diabetic Retinopathy using Deep Learning Algorithms for Segmentation to Facilitate Referral Recommendation for Test and Treatment Prediction
Automated surface segmentation is important and challenging in many medical image analysis applications. Recent deep learning based methods have been developed for various object segmentation tasks. Most of them are a classification based approach, e.g. U-net, which predicts the probability of being target object or background for each voxel. One problem of those methods is lacking of topology guarantee for segmented objects, and usually post processing is needed to infer the boundary surface of the object. In this paper, a novel model based on convolutional neural network (CNN) followed by a …
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Deep learning shows high potential for many medical image analysis tasks. Neural networks can work with full-size data without extensive preprocessing and feature generation and, thus, information loss. Recent work has shown that the morphological difference in specific brain regions can be found on MRI with the means of Convolution Neural Networks (CNN). However, interpretation of the existing models is based on a region of interest and can not be extended to voxel-wise image interpretation on a whole image. In the current work, we consider the classification task on a large-scale open-source…
