Analysis of Macula on Color Fundus Images Using Heightmap Reconstruction Through Deep Learning
University of Tehran（德黑兰大学）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）；Research Institute for Ophthalmology and Vision Science（眼科与视觉科学研究所）
Peyman Tahghighi, Reza A. Zoroofi, Sare Safi, Alireza Ramezani
备注：
arXiv admin note: substantial text overlap with arXiv:2009.01601
For medical diagnosis based on retinal images, a clear understanding of 3D structure is often required but due to the 2D nature of images captured, we cannot infer that information. However, by utilizing 3D reconstruction methods, we can recover the height information of the macula area on a fundus image which can be helpful for diagnosis and screening of macular disorders. Recent approaches have used shading information for heightmap prediction but their output was not accurate since they ignored the dependency between nearby pixels and only utilized shading information. Additionally, other methods were dependent on the availability of more than one image of the retina which is not available in practice. In this paper, motivated by the success of Conditional Generative Adversarial Networks(cGANs) and deeply supervised networks, we propose a novel architecture for the generator which enhances the details and the quality of output by progressive refinement and the use of deep supervision to reconstruct the height information of macula on a color fundus image. Comparisons on our own dataset illustrate that the proposed method outperforms all of the state-of-the-art methods in image translation and medical image translation on this particular task. Additionally, perceptual studies also indicate that the proposed method can provide additional information for ophthalmologists for diagnosis.
University of California, Berkeley（加利福尼亚大学伯克利分校）；International Computer Science Institute（国际计算机科学研究所）；University of California, Davis（加利福尼亚大学戴维斯分校）
Many diseases are classified based on human-defined rubrics that are prone to bias. Supervised neural networks can automate the grading of retinal fundus images, but require labor-intensive annotations and are restricted to the specific trained task. Here, we employed an unsupervised network with Non-Parametric Instance Discrimination (NPID) to grade age-related macular degeneration (AMD) severity using fundus photographs from the Age-Related Eye Disease Study (AREDS). Our unsupervised algorithm demonstrated versatility across different AMD classification schemes without retraining, and achiev…
Sun Yat-sen University（中山大学）；Baidu Inc.（百度公司）；Agency for Science, Technology and Research（新加坡科技研究局）；CONICET（国家科学技术研究委员会）；Medical University of Vienna（维也纳医科大学）；VUNO Inc.（VUNO公司）；Inserm（法国国家健康与医学研究院）；Univ Bretagne Occidentale（西布列塔尼大学）；Zasti（Zasti公司）
Age-related macular degeneration (AMD) is the leading cause of visual impairment among elderly in the world. Early detection of AMD is of great importance, as the vision loss caused by this disease is irreversible and permanent. Color fundus photography is the most cost-effective imaging modality to screen for retinal disorders. Cutting edge deep learning based algorithms have been recently developed for automatically detecting AMD from fundus images. However, there are still lack of a comprehensive annotated dataset and standard evaluation benchmarks. To deal with this issue, we set up the Au…
Aalto University School of Science（阿尔托大学理学院）；Digifundus Ltd.（Digifundus有限公司）；Central Finland Central Hospital（中芬兰中心医院）
Diabetes is a globally prevalent disease that can cause visible microvascular complications such as diabetic retinopathy and macular edema in the human eye retina, the images of which are today used for manual disease screening. This labor-intensive task could greatly benefit from automatic detection using deep learning technique. Here we present a deep learning system that identifies referable diabetic retinopathy comparably or better than presented in the previous studies, although we use only a small fraction of images (<1/4) in training but are aided with higher image resolutions. We also …
Multiscale Color Guided Attention Ensemble Classifier for Age-Related Macular Degeneration using Concurrent Fundus and Optical Coherence Tomography Images
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）；School of Medical Science and Technology, Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校医学科学与技术学院）；Department of Mathematics, Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校数学系）
Background: Patients with neovascular age-related macular degeneration (AMD) can avoid vision loss via certain therapy. However, methods to predict the progression to neovascular age-related macular degeneration (nvAMD) are lacking. Purpose: To develop and validate a deep learning (DL) algorithm to predict 1-year progression of eyes with no, early, or intermediate AMD to nvAMD, using color fundus photographs (CFP). Design: Development and validation of a DL algorithm. Methods: We trained a DL algorithm to predict 1-year progression to nvAMD, and used 10-fold cross-validation to evaluate this a…
University of Waterloo（滑铁卢大学）；Manipal Institute of Technology（马尼帕尔理工学院）
An overview of the applications of deep learning in ophthalmic diagnosis using retinal fundus images is presented. We also review various retinal image datasets that can be used for deep learning purposes. Applications of deep learning for segmentation of optic disk, blood vessels and retinal layer as well as detection of lesions are reviewed. Recent deep learning models for classification of diseases such as age-related macular degeneration, glaucoma,diabetic macular edema and diabetic retinopathy are also reported.
Google AI（谷歌人工智能）；Google（谷歌公司）；Rajavithi Hospital（拉差威提医院）；College of Medicine, Rangsit University（兰实大学医学院）；EyePACS LLC（EyePACS有限责任公司）；University of California（加利福尼亚大学）；Google DeepMind（谷歌DeepMind）；NIHR Biomedical Research Centre for Ophthalmology（国立卫生研究院眼科生物医学研究中心）；Moorfields Eye Hospital NHS Foundation Trust（摩尔菲尔德眼科医院英国国民保健制度基金会信托）；UCL Institute of Ophthalmology（伦敦大学学院眼科研究所）
Diabetic eye disease is one of the fastest growing causes of preventable blindness. With the advent of anti-VEGF (vascular endothelial growth factor) therapies, it has become increasingly important to detect center-involved diabetic macular edema (ci-DME). However, center-involved diabetic macular edema is diagnosed using optical coherence tomography (OCT), which is not generally available at screening sites because of cost and workflow constraints. Instead, screening programs rely on the detection of hard exudates in color fundus photographs as a proxy for DME, often resulting in high false p…
University of Tehran（德黑兰大学）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）
For screening, 3D shape of the eye retina often provides structural information and can assist ophthalmologists to diagnose diseases. However, fundus images which are one the most common screening modalities for retina diagnosis lack this information due to their 2D nature. Hence, in this work, we try to infer about this 3D information or more specifically its heights. Recent approaches have used shading information for reconstructing the heights but their output is not accurate since the utilized information is not sufficient. Additionally, other methods were dependent on the availability of …
Multiscale Softmax Cross Entropy for Fovea Localization on Color Fundus Photography
Institute of Imaging and Computer Vision, RWTH Aachen, Germany（成像与计算机视觉研究所，RWTH亚琛，德国）；Department of Ophthalmology, RWTH Aachen, Germany（眼科部门，RWTH亚琛，德国）
Age related macular degeneration (AMD) is one of the major causes for blindness in the elderly population. In this report, we propose deep learning based methods for retinal analysis using color fundus images for computer aided diagnosis of AMD. We leverage the recent state of the art deep networks for building a single fundus image based AMD classification pipeline. We also propose methods for the other directly relevant and auxiliary tasks such as lesions detection and segmentation, fovea detection and optic disc segmentation. We propose the use of generative adversarial networks (GANs) for …
EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University（首都医科大学附属北京同仁医院北京同仁眼科中心）；Liangxiang Hospital of Beijing Fangshan District, Capital Medical University（首都医科大学北京市房山区良乡医院）；The Third People's Hospital of Dalian（大连市第三人民医院）；National Clinical Research Center for Endocrine and Metabolic Diseases, The Second Xiangya Hospital of Central South University（中南大学湘雅二医院国家内分泌代谢病临床医学研究中心）；The Central Hospital of Baoji City（宝鸡市中心医院）；Wuxi No.2 People's Hospital, Affiliated Wuxi Clinical College of Nantong University（南通大学附属无锡临床学院无锡市第二人民医院）；Shenzhen Eye Hospital, Southern Medical University（南方医科大学深圳眼科医院）；Beijing Friendship Hospital, Capital Medical University（首都医科大学附属北京友谊医院）
Development of a Mobile Application for at-Home Analysis of Retinal Fundus Images
Division of Engineering Science（工程科学系）；University of Toronto（多伦多大学）；School of Information Technology（信息科技学院）；King Mongkut’s University of Technology Thonburi（泰国科技泰伦比大学）；Innovative Cognitive Computing Research Center（创新认知计算研究中心）；Chinese International School of Hong Kong（香港中文国际学校）
Universidade da Coruña（拉科鲁尼亚大学）；Instituto de Investigación Biomédica de A Coruña (INIBIC)（拉科鲁尼亚生物医学研究所）；Hospital Clínico San Carlos（圣卡洛斯临床医院）；Instituto de Investigación Sanitaria (IdISSC)（圣卡洛斯健康研究所）；Centro Internacional de Oftalmología Avanzada（国际高级眼科中心）
Age-related macular degeneration (AMD) is a degenerative disorder affecting the macula, a key area of the retina for visual acuity. Nowadays, it is the most frequent cause of blindness in developed countries. Although some promising treatments have been developed, their effectiveness is low in advanced stages. This emphasizes the importance of large-scale screening programs. Nevertheless, implementing such programs for AMD is usually unfeasible, since the population at risk is large and the diagnosis is challenging. All this motivates the development of automatic methods. In this sense, severa…
Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images
Ahsanullah University of Science and Technology（阿萨努拉科技大学）
Renmin University of China（中国人民大学）；Visionary Intelligence Ltd.（远景智能有限公司）；Peking Union Medical College Hospital（北京协和医院）；Chinese Academy of Medical Sciences（中国医学科学院）
This paper studies automated categorization of age-related macular degeneration (AMD) given a multi-modal input, which consists of a color fundus image and an optical coherence tomography (OCT) image from a specific eye. Previous work uses a traditional method, comprised of feature extraction and classifier training that cannot be optimized jointly. By contrast, we propose a two-stream convolutional neural network (CNN) that is end-to-end. The CNN's fusion layer is tailored to the need of fusing information from the fundus and OCT streams. For generating more multi-modal training instances, we…
University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）
Fundus photography has routinely been used to document the presence and severity of retinal degenerative diseases such as age-related macular degeneration (AMD), glaucoma, and diabetic retinopathy (DR) in clinical practice, for which the fovea and optic disc (OD) are important retinal landmarks. However, the occurrence of lesions, drusen, and other retinal abnormalities during retinal degeneration severely complicates automatic landmark detection and segmentation. Here we propose HBA-U-Net: a U-Net backbone enriched with hierarchical bottleneck attention. The network consists of a novel bottle…
JIS College of Engineering（JIS工程学院）；Kalyani Govt. Engg. College（卡利亚尼政府工程学院）
Digital Retinal Fundus Images helps to detect various ophthalmic diseases by detecting morphological changes in optical cup, optical disc and macula. Present work proposes a method for the authentication of medical images based on Discrete Wavelet Transformation (DWT) and Spread Spectrum. Proper selection of the Non Region of Interest (NROI) for watermarking is crucial, as the area under concern has to be the least required portion conveying any medical information. Proposed method discusses both the selection of least impact area and the blind watermarking technique. Watermark is embedded wit…
Interpretability of deep learning (DL) systems is gaining attention in medical imaging to increase experts' trust in the obtained predictions and facilitate their integration in clinical settings. We propose a deep visualization method to generate interpretability of DL classification tasks in medical imaging by means of visual evidence augmentation. The proposed method iteratively unveils abnormalities based on the prediction of a classifier trained only with image-level labels. For each image, initial visual evidence of the prediction is extracted with a given visual attribution technique. T…
Kaunas University of Technology（考纳斯理工大学）；Lithuanian University of Health Sciences（立陶宛健康科学大学）
This work aims to research an automatic method for detecting Age-related Macular Degeneration (AMD) lesions in RGB eye fundus images. For this, we align invasively obtained eye fundus contrast images (the "golden standard" diagnostic) to the RGB ones and use them to hand-annotate the lesions. This is done using our custom-made tool. Using the data, we train and test five different convolutional neural networks: a custom one to classify healthy and AMD-affected eye fundi, and four well-known networks: ResNet50, ResNet101, MobileNetV3, and UNet to segment (localize) the AMD lesions in the affect…
Replication studies are essential for validation of new methods, and are crucial to maintain the high standards of scientific publications, and to use the results in practice. We have attempted to replicate the main method in 'Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs' published in JAMA 2016; 316(22). We re-implemented the method since the source code is not available, and we used publicly available data sets. The original study used non-public fundus images from EyePACS and three hospitals in India for training.…
Centre Universitaire Condorcet（孔多塞大学中心）；Université de Bourgogne（勃艮第大学）
Diabetic retinopathy (DR) and age related macular degeneration (ARMD) are among the major causes of visual impairment worldwide. DR is mainly characterized by red spots, namely microaneurysms and bright lesions, specifically exudates whereas ARMD is mainly identified by tiny yellow or white deposits called drusen. Since exudates might be the only manifestation of the early diabetic retinopathy, there is an increase demand for automatic retinopathy diagnosis. Exudates and drusen may share similar appearances, thus discriminating between them is of interest to enhance screening performance. In t…
This paper experiments with the number of fully-connected layers in a deep convolutional neural network as applied to the classification of fundus retinal images. The images analysed corresponded to the ODIR 2019 (Peking University International Competition on Ocular Disease Intelligent Recognition) [9], which included images of various eye diseases (cataract, glaucoma, myopia, diabetic retinopathy, age-related macular degeneration (AMD), hypertension) as well as normal cases. This work focused on the classification of Normal, Cataract, AMD and Myopia. The feature extraction (convolutional) pa…
Fundus photography (FP) remains the primary imaging modality in screening various retinal diseases including age-related macular degeneration, diabetic retinopathy and glaucoma. FP allows the clinician to examine the ocular fundus structures such as the macula, the optic disc (OD) and retinal vessels, whose visibility and clarity in an FP image remain central to ensuring diagnostic accuracy, and hence determine the diagnostic quality (DQ). Images with low DQ, resulting from eye movement, improper illumination and other possible causes, should obviously be recaptured. However, the technician, o…
Faculty of Informatics, Kaunas University of Technology（信息学院，卡斯图技术大学）
Age-related macular degeneration (AMD) is one of the leading causes of irreversible vision impairment in people over the age of 60. This research focuses on semantic segmentation for AMD lesion detection in RGB fundus images, a non-invasive and cost-effective imaging technique. The results of the ADAM challenge - the most comprehensive AMD detection from RGB fundus images research competition and open dataset to date - serve as a benchmark for our evaluation. Taking the U-Net connectivity as a base of our framework, we evaluate and compare several approaches to improve the segmentation model's…
Cixi Institute of Biomedical Engineering, Chinese Academy of Sciences（中国科学院慈溪生物医学工程研究所）；Institute for Infocomm Research, Agency for Science, Technology and Research（新加坡科技研究局资讯通信研究院）；Shanghai University（上海大学）
Retinal fundus photographs have been used in the diagnosis of many ocular diseases such as glaucoma, pathological myopia, age-related macular degeneration and diabetic retinopathy. With the development of computer science, computer aided diagnosis has been developed to process and analyse the retinal images automatically. One of the challenges in the analysis is that the quality of the retinal image is often degraded. For example, a cataract in human lens will attenuate the retinal image, just as a cloudy camera lens which reduces the quality of a photograph. It often obscures the details in t…
Harnessing the power of longitudinal medical imaging for eye disease prognosis using Transformer-based sequence modeling
Weill Cornell Medicine（威尔康奈尔医学院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Washington University School of Medicine（华盛顿大学医学院）；Columbia University（哥伦比亚大学）；National Eye Institute（国家眼科研究所）；National Institutes of Health (NIH)（美国国立卫生研究院）；National Center for Biotechnology Information (NCBI)（国家生物技术信息中心）；National Library of Medicine (NLM)（国家医学图书馆）
Discriminating retinal microvascular and neuronal differences related to migraines: Deep Learning based Crossectional Study
Monash University（莫纳什大学）；University of New South Wales（新南威尔士大学）；School of Optometry & Vision Science（视光与视觉科学学院）；School of Computer Science and Engineering（计算机科学与工程学院）
University of Liverpool（利物浦大学）；Royal Liverpool University Hospital（皇家利物浦大学医院）
Prognostic models aim to predict the future course of a disease or condition and are a vital component of personalized medicine. Statistical models make use of longitudinal data to capture the temporal aspect of disease progression; however, these models require prior feature extraction. Deep learning avoids explicit feature extraction, meaning we can develop models for images where features are either unknown or impossible to quantify accurately. Previous prognostic models using deep learning with imaging data require annotation during training or only utilize a single time point. We propose …
A Deep Convolutional Neural Network-Based Novel Class Balancing for Imbalance Data Segmentation
Department of Computer Science, COMSATS University Islambad, Lahore Campus, Pakistan（计算机科学系，卡姆斯大学伊斯法罕，拉合尔校区，巴基斯坦）；Hamad Medical Corporation, Qatar（卡塔尔哈马姆医疗集团）；University of Stirling, UK（斯特林大学，英国）
Benchmarking Retinal Blood Vessel Segmentation Models for Cross-Dataset and Cross-Disease Generalization
University of Ibadan（伊巴丹大学）；African Institute for Mathematical Sciences（非洲数学科学研究所）；Hertie Institute for AI in Brain Health（赫蒂脑健康人工智能研究所）；University of Tübingen（蒂宾根大学）；University of Bern（伯尔尼大学）；University College London（伦敦大学学院）；Tübingen AI Center（蒂宾根人工智能中心）
