Fundus photography has routinely been used to document the presence and severity of retinal degenerative diseases such as age-related macular degeneration (AMD), glaucoma, and diabetic retinopathy (DR) in clinical practice, for which the fovea and optic disc (OD) are important retinal landmarks. However, the occurrence of lesions, drusen, and other retinal abnormalities during retinal degeneration severely complicates automatic landmark detection and segmentation. Here we propose HBA-U-Net: a U-Net backbone enriched with hierarchical bottleneck attention. The network consists of a novel bottleneck attention block that combines and refines self-attention, channel attention, and relative-position attention to highlight retinal abnormalities that may be important for fovea and OD segmentation in the degenerated retina. HBA-U-Net achieved state-of-the-art results on fovea detection across datasets and eye conditions (ADAM: Euclidean Distance (ED) of 25.4 pixels, REFUGE: 32.5 pixels, IDRiD: 32.1 pixels), on OD segmentation for AMD (ADAM: Dice Coefficient (DC) of 0.947), and on OD detection for DR (IDRiD: ED of 20.5 pixels). Our results suggest that HBA-U-Net may be well suited for landmark detection in the presence of a variety of retinal degenerative diseases.
Inception Institute of Artificial Intelligence(人工智能起源研究所)
Retinal fundus images are widely used for the clinical screening and diagnosis of eye diseases. However, fundus images captured by operators with various levels of experience have a large variation in quality. Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. However, due to the special optical beam of fundus imaging and structure of the retina, natural image enhancement methods cannot be utilized directly to address this. In this paper, we first analyze the ophthalmoscope imaging system and simulate a reliable degradation of major inf…
Tsinghua-Berkeley Shenzhen Institute(清华-伯克利深圳学院);University of California, Berkeley(加州大学伯克利分校);Affiliated Eye Hospital of Nanchang University(南昌大学附属眼科医院)
Retinopathy of prematurity (ROP) is an abnormal blood vessel development in the retina of a prematurely-born infant or an infant with low birth weight. ROP is one of the leading causes for infant blindness globally. Early detection of ROP is critical to slow down and avert the progression to vision impairment caused by ROP. Yet there is limited awareness of ROP even among medical professionals. Consequently, dataset for ROP is limited if ever available, and is in general extremely imbalanced in terms of the ratio between negative images and positive ones. In this study, we formulate the proble…
Eindhoven University of Technology(埃因霍温理工大学);Maastricht University Medical Center+(马斯特里赫特大学医学中心+)
Type 2 Diabetes (T2D) is a chronic metabolic disorder that can lead to blindness and cardiovascular disease. Information about early stage T2D might be present in retinal fundus images, but to what extent these images can be used for a screening setting is still unknown. In this study, deep neural networks were employed to differentiate between fundus images from individuals with and without T2D. We investigated three methods to achieve high classification performance, measured by the area under the receiver operating curve (ROC-AUC). A multi-target learning approach to simultaneously output r…
In recent years, several automatic segmentation methods have been proposed for blood vessels in retinal fundus images, ranging from using cheap and fast trainable filters to complicated neural networks and even deep learning. One example of a filted-based segmentation method is B-COSFIRE. In this approach the image filter is trained with example prototype patterns, to which the filter becomes selective by finding points in a Difference of Gaussian response on circles around the center with large intensity variation. In this paper we discuss and evaluate several of these vessel segmentation met…
Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images
Ahsanullah University of Science and Technology(阿萨努拉科技大学)
Fundus-Enhanced Disease-Aware Distillation Model for Retinal Disease Classification from OCT Images
The Hong Kong University of Science and Technology(香港科技大学);Guangdong Provincial Hospital of Integrated Traditional Chinese and Western Medicine(广东省中西医结合医院);Guangdong Weiren Meditech Co., Ltd(广东唯仁医疗科技有限公司)
Indian Institute of Technology Kharagpur(印度理工学院卡拉格普尔分校);Texas Instruments Inc.(德州仪器公司)
Vision impairment due to pathological damage of the retina can largely be prevented through periodic screening using fundus color imaging. However the challenge with large scale screening is the inability to exhaustively detect fine blood vessels crucial to disease diagnosis. In this work we present a computational imaging framework using deep and ensemble learning for reliable detection of blood vessels in fundus color images. An ensemble of deep convolutional neural networks is trained to segment vessel and non-vessel areas of a color fundus image. During inference, the responses of the indi…
Accurate measurements of abnormalities like Artery-Vein ratio and tortuosity in fundus images is an actively researched task. Most of the research seems to compute such features independently. However, in this work, we have devised a fully automated technique to measure any vascular abnormalities. This paper is a follow-up paper on vessel topology estimation and extraction, we use the extracted topology to perform A-V state-of-the-art Artery-Vein classification, AV ratio calculation, and vessel tortuosity measurement, all fully automated. Existing techniques tend to only work on the partial re…
Beijing University of Posts and Telecommunications(北京邮电大学)
Accurate retinal vessel segmentation is a challenging problem in color fundus image analysis. An automatic retinal vessel segmentation system can effectively facilitate clinical diagnosis and ophthalmological research. Technically, this problem suffers from various degrees of vessel thickness, perception of details, and contextual feature fusion. For addressing these challenges, a deep learning based method has been proposed and several customized modules have been integrated into the well-known encoder-decoder architecture U-net, which is mainly employed in medical image segmentation. Structu…
With the advancement of powerful image processing and machine learning techniques, CAD has become ever more prevalent in all fields of medicine including ophthalmology. Since optic disc is the most important part of retinal fundus image for glaucoma detection, this paper proposes a two-stage framework that first detects and localizes optic disc and then classifies it into healthy or glaucomatous. The first stage is based on RCNN and is responsible for localizing and extracting optic disc from a retinal fundus image while the second stage uses Deep CNN to classify the extracted disc into health…
Universidad Nacional de Colombia(哥伦比亚国立大学);Fundación Oftalmológica Nacional(国家眼科基金会);Universidad del Rosario(罗萨里奥大学)
Diabetic retinopathy (DR) is the result of a complication of diabetes affecting the retina. It can cause blindness, if left undiagnosed and untreated. An ophthalmologist performs the diagnosis by screening each patient and analyzing the retinal lesions via ocular imaging. In practice, such analysis is time-consuming and cumbersome to perform. This paper presents a model for automatic DR classification on eye fundus images. The approach identifies the main ocular lesions related to DR and subsequently diagnoses the illness. The proposed method follows the same workflow as the clinicians, provid…
University of California, Berkeley(加利福尼亚大学伯克利分校);International Computer Science Institute(国际计算机科学研究所);University of California, Davis(加利福尼亚大学戴维斯分校)
Many diseases are classified based on human-defined rubrics that are prone to bias. Supervised neural networks can automate the grading of retinal fundus images, but require labor-intensive annotations and are restricted to the specific trained task. Here, we employed an unsupervised network with Non-Parametric Instance Discrimination (NPID) to grade age-related macular degeneration (AMD) severity using fundus photographs from the Age-Related Eye Disease Study (AREDS). Our unsupervised algorithm demonstrated versatility across different AMD classification schemes without retraining, and achiev…
MM-Retinal: Knowledge-Enhanced Foundational Pretraining with Fundus Image-Text Expertise
School of Computer Science and Engineering, Southeast University, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, China(教育部新一代人工智能技术及其交叉应用重点实验室);Nanjing University of Science and Technology, Nanjing, China(南京理工大学);Agency for Science, Technology and Research (A*STAR), Singapore(新加坡科技研究局)
Artery and vein (AV) classification of retinal images is a key to necessary tasks, such as automated measurement of arteriolar-to-venular diameter ratio (AVR). This paper comprehensively reviews the state-of-the art in AV classification methods. To improve on previous methods, a new Local Bi- nary Pattern-based method (LBP) is proposed. Beside its simplicity, LBP is robust against low contrast and low quality fundus images; and it helps the process by including additional AV texture and shape information. Experimental results compare the performance of the new method with the state-of-the art;…
Ultra-widefield (UWF) imaging is a promising modality that captures a larger retinal field of view compared to traditional fundus photography. Previous studies showed that deep learning (DL) models are effective for detecting retinal disease in UWF images, but primarily considered individual diseases under less-than-realistic conditions (excluding images with other diseases, artefacts, comorbidities, or borderline cases; and balancing healthy and diseased images) and did not systematically investigate which regions of the UWF images are relevant for disease detection. We first improve on the s…
University of Waterloo(滑铁卢大学);Manipal Institute of Technology(马尼帕尔理工学院)
An overview of the applications of deep learning in ophthalmic diagnosis using retinal fundus images is presented. We also review various retinal image datasets that can be used for deep learning purposes. Applications of deep learning for segmentation of optic disk, blood vessels and retinal layer as well as detection of lesions are reviewed. Recent deep learning models for classification of diseases such as age-related macular degeneration, glaucoma,diabetic macular edema and diabetic retinopathy are also reported.
Purpose: Convolutional neural networks can be trained to detect various conditions or patient traits based on retinal fundus photographs, some of which, such as the patient sex, are invisible to the expert human eye. Here we propose a methodology for explainable classification of fundus images to uncover the mechanism(s) by which CNNs successfully predict the labels. We used patient sex as a case study to validate our proposed methodology. Approach: First, we used a set of 4746 fundus images, including training, validation and test partitions, to fine-tune a pre-trained CNN on the sex classifi…
We explore the value of weak labels in learning transferable representations for medical images. Compared to hand-labeled datasets, weak or inexact labels can be acquired in large quantities at significantly lower cost and can provide useful training signals for data-hungry models such as deep neural networks. We consider weak labels in the form of pseudo-labels and propose a semi-weakly supervised contrastive learning (SWCL) framework for representation learning using semi-weakly annotated images. Specifically, we train a semi-supervised model to propagate labels from a small dataset consisti…
Automated Computer Aided diagnostic tools can be used for the early detection of glaucoma to prevent irreversible vision loss. In this work, we present a Multi-task Convolutional Neural Network (CNN) that jointly segments the Optic Disc (OD), Optic Cup (OC) and predicts the presence of glaucoma in color fundus images. The CNN utilizes a combination of image appearance features and structural features obtained from the OD-OC segmentation to obtain a robust prediction. The use of fewer network parameters and the sharing of the CNN features for multiple related tasks ensures the good generalizabi…
Despite its high prevalence, anemia is often undetected due to the invasiveness and cost of screening and diagnostic tests. Though some non-invasive approaches have been developed, they are less accurate than invasive methods, resulting in an unmet need for more accurate non-invasive methods. Here, we show that deep learning-based algorithms can detect anemia and quantify several related blood measurements using retinal fundus images both in isolation and in combination with basic metadata such as patient demographics. On a validation dataset of 11,388 patients from the UK Biobank, our algorit…
Development of a Mobile Application for at-Home Analysis of Retinal Fundus Images
Division of Engineering Science(工程科学系);University of Toronto(多伦多大学);School of Information Technology(信息科技学院);King Mongkut’s University of Technology Thonburi(泰国科技泰伦比大学);Innovative Cognitive Computing Research Center(创新认知计算研究中心);Chinese International School of Hong Kong(香港中文国际学校)
Carnegie Mellon University(卡内基梅隆大学);INESC TEC;Faculty of Engineering, University of Porto(波尔图大学工程学院)
We propose a pixel color amplification theory and family of enhancement methods to facilitate segmentation tasks on retinal images. Our novel re-interpretation of the image distortion model underlying dehazing theory shows how three existing priors commonly used by the dehazing community and a novel fourth prior are related. We utilize the theory to develop a family of enhancement methods for retinal images, including novel methods for whole image brightening and darkening. We show a novel derivation of the Unsharp Masking algorithm. We evaluate the enhancement methods as a pre-processing step…
A Disease-Specific Foundation Model Using Over 100K Fundus Images: Release and Validation for Abnormality and Multi-Disease Classification on Downstream Tasks
Seoul National University(首尔大学);Seoul National University Hospital(首尔大学附属医院)
JIS College of Engineering(JIS工程学院);Kalyani Govt. Engg. College(卡利亚尼政府工程学院)
Digital Retinal Fundus Images helps to detect various ophthalmic diseases by detecting morphological changes in optical cup, optical disc and macula. Present work proposes a method for the authentication of medical images based on Discrete Wavelet Transformation (DWT) and Spread Spectrum. Proper selection of the Non Region of Interest (NROI) for watermarking is crucial, as the area under concern has to be the least required portion conveying any medical information. Proposed method discusses both the selection of least impact area and the blind watermarking technique. Watermark is embedded wit…
The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院);University of Science and Technology of China(中国科学技术大学)
Diabetic retinopathy is the most important complication of diabetes. Early diagnosis of retinal lesions helps to avoid visual loss or blindness. Due to high-resolution and small-size lesion regions, applying existing methods, such as U-Nets, to perform segmentation on fundus photography is very challenging. Although downsampling the input images could simplify the problem, it loses detailed information. Conducting patch-level analysis helps reaching fine-scale segmentation yet usually leads to misunderstanding as the lack of context information. In this paper, we propose an efficient network t…
Siksha ‘O’ Anusandhan (Deemed to be University)(锡克沙奥阿努山丹大学(被认定为大学));ITER, Siksha ‘O’ Anusandhan (Deemed to be University)(锡克沙奥阿努山丹大学(被认定为大学)ITER学院);Center for the Internet of Things, ITER, Siksha ‘O’ Anusandhan (Deemed to be University)(锡克沙奥阿努山丹大学(被认定为大学)ITER学院物联网中心)
Extracting blood vessels from retinal fundus images plays a decisive role in diagnosing the progression in pertinent diseases. In medical image analysis, vessel extraction is a semantic binary segmentation problem, where blood vasculature needs to be extracted from the background. Here, we present an image enhancement technique based on the morphological preprocessing coupled with a scaled U-net architecture. Despite a relatively less number of trainable network parameters, the scaled version of U-net architecture provides better performance compare to other methods in the domain. We validated…
Indian Institute of Technology Kharagpur(印度理工学院卡拉格普尔分校)
Automated segmentation of retinal blood vessels in label-free fundus images entails a pivotal role in computed aided diagnosis of ophthalmic pathologies, viz., diabetic retinopathy, hypertensive disorders and cardiovascular diseases. The challenge remains active in medical image analysis research due to varied distribution of blood vessels, which manifest variations in their dimensions of physical appearance against a noisy background. In this paper we formulate the segmentation challenge as a classification task. Specifically, we employ unsupervised hierarchical feature learning using ensembl…
MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretraining
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Department of Ophthalmology, The First Affiliated Hospital of Nanjing Medical University(南京医科大学第一附属医院眼科部);School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);Institute of High-Performance Computing, Agency for Science, Technology and Research(科技研究局高性能计算研究所)
Disentangling representations of retinal images with generative models
Hertie Institute for AI in Brain Health, Faculty of Medicine, University of Tübingen(人工智能与脑健康赫特研究所,医学院,图宾根大学);Tübingen AI Center, Tübingen, Germany(图宾根人工智能中心,图宾根,德国);Department of Diabetes, Endocrinology, Nutritional Medicine and Metabolism UDEM, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland(糖尿病、内分泌学、营养医学和代谢学部门UDEM,伯尔尼大学医院,伯尔尼大学,伯尔尼,瑞士);Department of Computer Science, University of Tübingen, Tübingen, Germany(计算机科学系,图宾根大学,图宾根,德国)
Early detection of retinal diseases is one of the most important means of preventing partial or permanent blindness in patients. In this research, a novel multi-label classification system is proposed for the detection of multiple retinal diseases, using fundus images collected from a variety of sources. First, a new multi-label retinal disease dataset, the MuReD dataset, is constructed, using a number of publicly available datasets for fundus disease classification. Next, a sequence of post-processing steps is applied to ensure the quality of the image data and the range of diseases, present …
Google Research(谷歌研究院);Google DeepMind(谷歌DeepMind);NIHR Biomedical Research Centre for Ophthalmology(NIHR眼科生物医学研究中心);Moorfields Eye Hospital NHS Foundation Trust(摩尔菲尔德眼科医院NHS基金会信托);UCL Institute of Ophthalmology(伦敦大学学院眼科研究所)
Refractive error, one of the leading cause of visual impairment, can be corrected by simple interventions like prescribing eyeglasses. We trained a deep learning algorithm to predict refractive error from the fundus photographs from participants in the UK Biobank cohort, which were 45 degree field of view images and the AREDS clinical trial, which contained 30 degree field of view images. Our model use the "attention" method to identify features that are correlated with refractive error. Mean absolute error (MAE) of the algorithm's prediction compared to the refractive error obtained in the AR…
Integrating Deep Learning with Fundus and Optical Coherence Tomography for Cardiovascular Disease Prediction
University of Leeds(利兹大学);University of Manchester(曼彻斯特大学);NIHR Manchester Biomedical Research Centre(NIHR曼彻斯特生物医学研究中心);Manchester Academic Health Science Centre(曼彻斯特学术健康科学中心);Christabel Pankhurst Institute(克里斯塔贝尔·潘克赫斯特研究所)
Manarat International University(玛纳拉特国际大学);Bangladesh University of Engineering and Technology(孟加拉国工程技术大学)
Diabetic Retinopathy (DR) is a constantly deteriorating disease, being one of the leading causes of vision impairment and blindness. Subtle distinction among different grades and existence of many significant small features make the task of recognition very challenging. In addition, the present approach of retinopathy detection is a very laborious and time-intensive task, which heavily relies on the skill of a physician. Automated detection of diabetic retinopathy is essential to tackle these problems. Early-stage detection of diabetic retinopathy is also very important for diagnosis, which ca…
Graduate School in Advanced Optical Technologies (SAOT)(高级光学技术研究生院(SAOT))
The acquisition of high-resolution retinal fundus images with a large field of view (FOV) is challenging due to technological, physiological and economic reasons. This paper proposes a fully automatic framework to reconstruct retinal images of high spatial resolution and increased FOV from multiple low-resolution images captured with non-mydriatic, mobile and video-capable but low-cost cameras. Within the scope of one examination, we scan different regions on the retina by exploiting eye motion conducted by a patient guidance. Appropriate views for our mosaicing method are selected based on op…
Objective Automatic artery/vein (A/V) segmentation from fundus images is required to track blood vessel changes occurring with many pathologies including retinopathy and cardiovascular pathologies. One of the clinical measures that quantifies vessel changes is the arterio-venous ratio (AVR) which represents the ratio between artery and vein diameters. This measure significantly depends on the accuracy of vessel segmentation and classification into arteries and veins. This paper proposes a fast, novel method for semantic A/V segmentation combining deep learning and graph propagation. Methods A …
Monash University(莫纳什大学);Zhongshan Ophthalmic Center, Sun Yat-sen University(中山大学中山眼科中心);University of Melbourne(墨尔本大学)
Recent studies have validated the association between cardiovascular disease (CVD) risk and retinal fundus images. Combining deep learning (DL) and portable fundus cameras will enable CVD risk estimation in various scenarios and improve healthcare democratization. However, there are still significant issues to be solved. One of the top priority issues is the different camera differences between the databases for research material and the samples in the production environment. Most high-quality retinography databases ready for research are collected from high-end fundus cameras, and there is a …
Replication studies are essential for validation of new methods, and are crucial to maintain the high standards of scientific publications, and to use the results in practice. We have attempted to replicate the main method in 'Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs' published in JAMA 2016; 316(22). We re-implemented the method since the source code is not available, and we used publicly available data sets. The original study used non-public fundus images from EyePACS and three hospitals in India for training.…
Robert Bosch GmbH(罗伯特·博世有限公司);TU Dortmund University(多特蒙德工业大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)
Convolutional neural networks (CNNs) show impressive performance for image classification and detection, extending heavily to the medical image domain. Nevertheless, medical experts are sceptical in these predictions as the nonlinear multilayer structure resulting in a classification outcome is not directly graspable. Recently, approaches have been shown which help the user to understand the discriminative regions within an image which are decisive for the CNN to conclude to a certain class. Although these approaches could help to build trust in the CNNs predictions, they are only slightly sho…
Indian Institute of Science(印度科学学院);University of Southern California(南加州大学)
Optic disc (OD) and optic cup (OC) are regions of prominent clinical interest in a retinal fundus image. They are the primary indicators of a glaucomatous condition. With the advent and success of deep learning for healthcare research, several approaches have been proposed for the segmentation of important features in retinal fundus images. We propose a novel approach for the simultaneous segmentation of the OD and OC using a residual encoder-decoder network (REDNet) based regional convolutional neural network (RCNN). The RED-RCNN is motivated by the Mask RCNN (MRCNN). Performance comparisons …
The visual analysis of retina and of its vascular characteristics is important in the diagnosis and monitoring of diseases of visual perception. In the related medical diagnoses, the digital processing of the fundus images is used to obtain the segmentation of retinal vessels. However, an image segmentation is often requiring methods based on peculiar or complex algorithms: in this paper we will show some alternative approaches obtained by applying freely available tools to enhance, without a specific segmentation, the images of the fundus of the eye. We will see in particular, that combining …
A Deep Learning-Based Unified Framework for Red Lesions Detection on Retinal Fundus Images
Computer and Information Science College, King Saud University(沙特王后大学计算机与信息科学学院);Department of Ophthalmology, College of Medicine, Princess Nourah bint Abdulrahman University(普里丝特努尔·阿卜杜勒拉赫曼大学医学院眼科系)
Artificial Intelligence in Assessing Cardiovascular Diseases and Risk Factors via Retinal Fundus Images: A Review of the Last Decade
Nikookari Eye Hospital(尼库卡里眼科医院);Tabriz University of Medical Sciences(大不里士医科大学);Cardiovascular Research Center(心血管研究中心);Research Center for Evidence‑Based Medicine(循证医学研究中心);Iranian EBM Centre: A Joanna Briggs Institute Affiliated Group(伊朗循证医学中心:乔安娜·布里格斯研究所附属机构);Tabriz Islamic Art University(大不里士伊斯兰艺术大学);Institute for Intelligent Systems Research and Innovation (IISRI)(智能系统研究与创新研究所(IISRI));Deakin University(迪肯大学)
A Hybrid Fully Convolutional CNN-Transformer Model for Inherently Interpretable Disease Detection from Retinal Fundus Images
Hertie Institute for AI in Brain Health, University of Tübingen, Germany(人工智能与脑健康赫尔特研究所,图宾根大学,德国);Tübingen AI Center, University of Tübingen, Germany(图宾根人工智能中心,图宾根大学,德国)
University of Tehran(德黑兰大学);Shahid Beheshti University of Medical Sciences(沙希德·贝赫什提医科大学);Research Institute for Ophthalmology and Vision Science(眼科与视觉科学研究所)
For medical diagnosis based on retinal images, a clear understanding of 3D structure is often required but due to the 2D nature of images captured, we cannot infer that information. However, by utilizing 3D reconstruction methods, we can recover the height information of the macula area on a fundus image which can be helpful for diagnosis and screening of macular disorders. Recent approaches have used shading information for heightmap prediction but their output was not accurate since they ignored the dependency between nearby pixels and only utilized shading information. Additionally, other m…
Fundus image captures rear of an eye, and which has been studied for the diseases identification, classification, segmentation, generation, and biological traits association using handcrafted, conventional, and deep learning methods. In biological traits estimation, most of the studies have been carried out for the age prediction and gender classification with convincing results. However, the current study utilizes the cutting-edge deep learning (DL) algorithms to estimate biological traits in terms of age and gender together with associating traits to retinal visuals. For the traits associati…
Explainable Artificial Intelligence (AI) in the form of an interpretable and semiautomatic approach to stage grading ocular pathologies such as Diabetic retinopathy, Hypertensive retinopathy, and other retinopathies on the backdrop of major systemic diseases. The experimental study aims to evaluate an explainable staged grading process without using deep Convolutional Neural Networks (CNNs) directly. Many current CNN-based deep neural networks used for diagnosing retinal disorders might have appreciable performance but fail to pinpoint the basis driving their decisions. To improve these decisi…