Ocular diseases, including diabetic retinopathy and glaucoma, present a significant public health challenge due to their high prevalence and potential for causing vision impairment. Early and accurate diagnosis is crucial for effective treatment and management. In recent years, deep learning models have emerged as powerful tools for analysing medical images, such as retina imaging. However, challenges persist in model relibability and uncertainty estimation, which are critical for clinical decision-making. This study leverages the probabilistic framework of Generative Flow Networks (GFlowNets) to learn the posterior distribution over latent discrete dropout masks for the classification and analysis of ocular diseases using fundus images. We develop a robust and generalizable method that utilizes GFlowOut integrated with ResNet18 and ViT models as the backbone in identifying various ocular conditions. This study employs a unique set of dropout masks - none, random, bottomup, and topdown - to enhance model performance in analyzing these fundus images. Our results demonstrate that our learnable probablistic latents significantly improves accuracy, outperforming the traditional dropout approach. We utilize a gradient map calculation method, Grad-CAM, to assess model explainability, observing that the model accurately focuses on critical image regions for predictions. The integration of GFlowOut in neural networks presents a promising advancement in the automated diagnosis of ocular diseases, with implications for improving clinical workflows and patient outcomes.
SSVT: Self-Supervised Vision Transformer For Eye Disease Diagnosis Based On Fundus Images
Beihang University(北京航空航天大学);Beijing Tongren Hospital, Capital Medical University(首都医科大学附属北京同仁医院);The University of Melbourne(墨尔本大学);University of Cambridge(剑桥大学);Imperial College London(帝国理工学院)
Clinical diagnosis of the eye is performed over multifarious data modalities including scalar clinical labels, vectorized biomarkers, two-dimensional fundus images, and three-dimensional Optical Coherence Tomography (OCT) scans. Clinical practitioners use all available data modalities for diagnosing and treating eye diseases like Diabetic Retinopathy (DR) or Diabetic Macular Edema (DME). Enabling usage of machine learning algorithms within the ophthalmic medical domain requires research into the relationships and interactions between all relevant data over a treatment period. Existing datasets…
Harnessing the power of longitudinal medical imaging for eye disease prognosis using Transformer-based sequence modeling
Weill Cornell Medicine(威尔康奈尔医学院);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Washington University School of Medicine(华盛顿大学医学院);Columbia University(哥伦比亚大学);National Eye Institute(国家眼科研究所);National Institutes of Health (NIH)(美国国立卫生研究院);National Center for Biotechnology Information (NCBI)(国家生物技术信息中心);National Library of Medicine (NLM)(国家医学图书馆)
University of Toronto(多伦多大学);The Hospital for Sick Children (SickKids)(病童医院(SickKids));Vector Institute(向量研究所)
Traditional datasets for the radiological diagnosis tend to only provide the radiology image alongside the radiology report. However, radiology reading as performed by radiologists is a complex process, and information such as the radiologist's eye-fixations over the course of the reading has the potential to be an invaluable data source to learn from. Nonetheless, the collection of such data is expensive and time-consuming. This leads to the question of whether such data is worth the investment to collect. This paper utilizes the recently published Eye-Gaze dataset to perform an exhaustive st…
PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur(贾巴尔普尔PDPM印度信息技术设计与制造学院)
It is feasible to recognize the presence and seriousness of eye disease by investigating the progressions in retinal biological structure. Fundus examination is a diagnostic procedure to examine the biological structure and anomaly of the eye. Ophthalmic diseases like glaucoma, diabetic retinopathy, and cataract are the main reason for visual impairment around the world. Ocular Disease Intelligent Recognition (ODIR-5K) is a benchmark structured fundus image dataset utilized by researchers for multi-label multi-disease classification of fundus images. This work presents a discriminative kernel …
University of Toronto(多伦多大学);The Hospital for Sick Children (SickKids)(病童医院(SickKids));Vector Institute(向量研究所)
As deep learning is widely used in the radiology field, the explainability of such models is increasingly becoming essential to gain clinicians' trust when using the models for diagnosis. In this research, three experiment sets were conducted with a U-Net architecture to improve the classification performance while enhancing the heatmaps corresponding to the model's focus through incorporating heatmap generators during training. All of the experiments used the dataset that contained chest radiographs, associated labels from one of the three conditions ("normal", "congestive heart failure (CHF)…
In this work, we present RadioTransformer, a novel visual attention-driven transformer framework, that leverages radiologists' gaze patterns and models their visuo-cognitive behavior for disease diagnosis on chest radiographs. Domain experts, such as radiologists, rely on visual information for medical image interpretation. On the other hand, deep neural networks have demonstrated significant promise in similar tasks even where visual interpretation is challenging. Eye-gaze tracking has been used to capture the viewing behavior of domain experts, lending insights into the complexity of visual …
School of Automation Science and Electrical Engineering, Beihang University (BUAA)(北京航空航天大学自动化科学与电气工程学院);Khalifa University of Science & Technology(哈利法科技大学);School of Computer and Communication Engineering, University of Science & Technology Beijing(北京科技大学计算机与通信工程学院);School of Electrical Engineering and Intelligentization, Dongguan University of Technology(东莞理工学院电气工程与智能化学院)
This paper proposes an automated method for the segmentation and extraction of the posterior segment of the human eye, including the vitreous, retina, choroid, and sclera compartments, using multi-vendor optical coherence tomography (OCT) scans. The proposed method works in two phases. First extracts the retinal pigment epithelium (RPE) layer by applying the adaptive thresholding technique to identify the retina-choroid junction. Then, it exploits the structure tensor guided approach to extract the inner limiting membrane (ILM) and the choroidal stroma (CS) layers, locating the vitreous-retina…
Accurate segmentation of medical images is essential for diagnosis and treatment of diseases. These problems are solved by highly complex models, such as deep networks (DN), requiring a large amount of labeled data for training. Thereby, many DNs possess task- or imaging modality specific architectures with a decision-making process that is often hard to explain and interpret. Here, we propose a framework that embeds existing DNs into a low-dimensional subspace induced by the learnable explicit feature map (LEFM) layer. Compared to the existing DN, the framework adds one hyperparameter and onl…
Visulytix Ltd(Visulytix有限公司);King’s College Hospital NHS Foundation Trust(国王学院医院NHS基金会信托);Cardiff University(卡迪夫大学);Ophthalmica, Ophthalmology and Microsurgery Institute(Ophthalmica眼科与显微外科研究所)
Purpose: To evaluate Pegasus-OCT, a clinical decision support software for the identification of features of retinal disease from macula OCT scans, across heterogenous populations involving varying patient demographics, device manufacturers, acquisition sites and operators. Methods: 5,588 normal and anomalous macular OCT volumes (162,721 B-scans), acquired at independent centres in five countries, were processed using the software. Results were evaluated against ground truth provided by the dataset owners. Results: Pegasus-OCT performed with AUROCs of at least 98% for all datasets in the detec…
IBM Research(IBM研究);Almaden Research Center(阿尔马登研究中心);Department of Radiology and Imaging Sciences, Emory University(埃默里大学放射与影像科学系);Emory University(埃默里大学)
We developed a rich dataset of Chest X-Ray (CXR) images to assist investigators in artificial intelligence. The data were collected using an eye tracking system while a radiologist reviewed and reported on 1,083 CXR images. The dataset contains the following aligned data: CXR image, transcribed radiology report text, radiologist's dictation audio and eye gaze coordinates data. We hope this dataset can contribute to various areas of research particularly towards explainable and multimodal deep learning / machine learning methods. Furthermore, investigators in disease classification and localiza…
University of Bournemouth(伯恩茅斯大学);École de Technolgie Superieure de Montréal(蒙特利尔高等技术学院);Diagnos Inc.(Diagnos公司);CRCHUM University of Montreal Hospital Centre(蒙特利尔大学医院研究中心)
Assessing the degree of disease severity in biomedical images is a task similar to standard classification but constrained by an underlying structure in the label space. Such a structure reflects the monotonic relationship between different disease grades. In this paper, we propose a straightforward approach to enforce this constraint for the task of predicting Diabetic Retinopathy (DR) severity from eye fundus images based on the well-known notion of Cost-Sensitive classification. We expand standard classification losses with an extra term that acts as a regularizer, imposing greater penaltie…
The visual analysis of retina and of its vascular characteristics is important in the diagnosis and monitoring of diseases of visual perception. In the related medical diagnoses, the digital processing of the fundus images is used to obtain the segmentation of retinal vessels. However, an image segmentation is often requiring methods based on peculiar or complex algorithms: in this paper we will show some alternative approaches obtained by applying freely available tools to enhance, without a specific segmentation, the images of the fundus of the eye. We will see in particular, that combining …
According to multiple authoritative authorities, including the World Health Organization, vision-related impairments and disorders are becoming a significant issue. According to a recent report, one of the leading causes of irreversible blindness in persons over the age of 50 is delayed cataract treatment. A cataract is a cloudy spot in the eye's lens that causes visual loss. Cataracts often develop slowly and consequently result in difficulty in driving, reading, and even recognizing faces. This necessitates the development of rapid and dependable diagnosis and treatment solutions for ocular …
Fundus-Enhanced Disease-Aware Distillation Model for Retinal Disease Classification from OCT Images
The Hong Kong University of Science and Technology(香港科技大学);Guangdong Provincial Hospital of Integrated Traditional Chinese and Western Medicine(广东省中西医结合医院);Guangdong Weiren Meditech Co., Ltd(广东唯仁医疗科技有限公司)
A Disease-Specific Foundation Model Using Over 100K Fundus Images: Release and Validation for Abnormality and Multi-Disease Classification on Downstream Tasks
Seoul National University(首尔大学);Seoul National University Hospital(首尔大学附属医院)
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Glaucoma is one of the most severe eye diseases, characterized by rapid progression and leading to irreversible blindness. It is often the case that diagnostics is carried out when one's sight has already significantly degraded due to the lack of noticeable symptoms at early stage of the disease. Regular glaucoma screenings of the population shall improve early-stage detection, however the desirable frequency of etymological checkups is often not feasible due to the excessive load imposed by manual diagnostics on limited number of specialists. Considering the basic methodology to detect glauco…
Reliable Multimodality Eye Disease Screening via Mixture of Student's t Distributions
Sichuan University(四川大学);Shantou University(汕头大学);The Chinese University of Hong Kong(香港中文大学);Institute of High Performance Computing, A*STAR(新加坡科技研究局高性能计算研究所)
Monta Vista High School(蒙塔维斯塔高中);Hanoi-Amsterdam High School(河内-阿姆斯特丹高中);Churchill Fulshear High School(丘吉尔·福尔瑟高中);Norfolk Academy(诺福克学院);John H. Guyer High School(约翰·H·盖尔高中);Collège Sainte-Anne(圣安妮学院)
Today, more than 12 million people over the age of 40 suffer from ocular diseases. Most commonly, older patients are susceptible to age related macular degeneration, an eye disease that causes blurring of the central vision due to the deterioration of the retina. The former can only be detected through complex and expensive imaging software, markedly a visual field test; this leaves a significant population with untreated eye disease and holds them at risk for complete vision loss. The use of machine learning algorithms has been proposed for treating eye disease. However, the development of th…
FusionFM: Fusing Eye-specific Foundational Models for Optimized Ophthalmic Diagnosis
Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore(新加坡国立大学 Yong Loo Lin 医学院眼科系);Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore, Singapore(新加坡国立大学 Yong Loo Lin 医学院创新与精准眼科健康中心);Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore(新加坡眼科研究院);Ophthalmology and Visual Science Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore(杜克-国立新加坡大学医学学校眼科与视觉科学学术临床项目);Centre for Medical Image Computing, University College London, London, United Kingdom(伦敦大学学院医学图像计算中心);Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, Shantou, China(汕尾国际眼科中心,汕尾大学及香港中文大学);Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland(苏黎世州立医院眼科部);Spross Research Institute, Zurich, Switzerland(苏黎世 Spross 研究院);Institute of High Performance Computing, A*STAR, Singapore(新加坡 A*STAR 高性能计算研究院);NIHR Biomedical Research Centre, Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom(英国国家健康服务基金会 Moorfields 眼科医院生物医学研究中心);Institute of Ophthalmology, University College London, London, United Kingdom(伦敦大学学院眼科研究院)
Tianjin University(天津大学);Kunming University of Science and Technology(昆明理工大学);Tianjin Dementia Institute(天津痴呆研究所);Tianjin Huanhu Hospital(天津环湖医院)
Alzheimer's Disease (AD) causes a continuous decline in memory, thinking, and judgment. Traditional diagnoses are usually based on clinical experience, which is limited by some realistic factors. In this paper, we focus on exploiting deep learning techniques to diagnose AD based on eye-tracking behaviors. Visual attention, as typical eye-tracking behavior, is of great clinical value to detect cognitive abnormalities in AD patients. To better analyze the differences in visual attention between AD patients and normals, we first conduct a 3D comprehensive visual task on a non-invasive eye-trackin…
Jadavpur University(贾达布尔大学);IIT Madras(印度理工学院马德拉斯分校);IIT Jodhpur(印度理工学院焦特布尔分校);Inception Institute of AI(Inception人工智能研究院);Monash University(莫纳什大学);Airdoc-Monash Research(Airdoc-莫纳什研究中心)
Convolutional Neural Network models have successfully detected retinal illness from optical coherence tomography (OCT) and fundus images. These CNN models frequently rely on vast amounts of labeled data for training, difficult to obtain, especially for rare diseases. Furthermore, a deep learning system trained on a data set with only one or a few diseases cannot detect other diseases, limiting the system's practical use in disease identification. We have introduced an unsupervised approach for detecting anomalies in retinal images to overcome this issue. We have proposed a simple, memory effic…
Robust Rigid and Non-Rigid Medical Image Registration Using Learnable Edge Kernels
Department of Mathematics, University of Innsbruck(因斯布鲁克大学数学系);Department of Radiology, Medical University of Innsbruck(因斯布鲁克医学大学放射学系);Neuroimaging Research Core Facility, Medical University of Innsbruck(因斯布鲁克医学大学神经影像研究核心设施)
Collaborative Attention and Consistent-Guided Fusion of MRI and PET for Alzheimer's Disease Diagnosis
School of Software Yunnan University(软件学院 云南大学);Department of Computer Science The University of Sheffield(计算机科学系 剑桥大学);Department of Computing Xian JiaoTong-Liverpool University(计算系 西交利物浦大学)
RET-CLIP: A Retinal Image Foundation Model Pre-trained with Clinical Diagnostic Reports
Beijing Institute of Technology(北京理工大学);Beijing Tongren Hospital(北京同仁医院);Capital Medical University(首都医科大学);Beijing Institute of Ophthalmology(北京眼科研究所)
Multi-scale Spatio-temporal Transformer-based Imbalanced Longitudinal Learning for Glaucoma Forecasting from Irregular Time Series Images
Zhejiang Lab(之江实验室);The Chinese University of Hong Kong(香港中文大学);Beijing Institute of Ophthalmology(北京眼科研究所);Beijing Tongren Eye Center(北京同仁眼科中心);Beijing Tongren Hospital(北京同仁医院);Capital Medical University(首都医科大学);Beijing Key Laboratory of Ophthalmology and Visual Sciences(北京市眼科与视觉科学重点实验室)
GazeLT: Visual attention-guided long-tailed disease classification in chest radiographs
Department of Biomedical Informatics, Stony Brook University, NY, US(斯通比克大学生物医学信息学系);Department of Radiology, Columbia University, NY, US(哥伦比亚大学放射学系);Department of Computer Science, Stony Brook University, NY, US(斯通比克大学计算机科学系)
Confidence-aware multi-modality learning for eye disease screening
Sichuan University(四川大学);College of Computer Science, Sichuan University(四川大学计算机学院);Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong(汕头大学与香港中文大学联合汕头国际眼科中心);Medical College, Shantou University(汕头大学医学院);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Institute of High Performance Computing, Agency for Science, Technology and Research(新加坡科技研究局高性能计算研究院)
An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care
Singapore Eye Research Institute, Singapore National Eye Centre(新加坡眼科研究所,新加坡国家眼科中心);Institute of High Performance Computing, Agency of Science, Technology and Research(高性能计算研究所,新加坡科技研究局);University of Pennsylvania(宾夕法尼亚大学);Duke-NUS Medical School(杜克-新加坡国立大学医学院);Columbia University(哥伦比亚大学);National University of Singapore(新加坡国立大学);Chang Gung Memorial Hospital(长庚纪念医院);Chang Gung University(长庚大学);Shanghai Eye Disease Prevention & Treatment Center, Shanghai Eye Hospital(上海市眼病防治中心,上海市眼医院)
SimulaMet;Oslo Metropolitan University(奥斯陆城市大学);Odense University Hospital(欧登塞大学医院);Ifocus;Oslo University Hospital(奥斯陆大学医院)
Dry eye disease (DED) has a prevalence of between 5 and 50\%, depending on the diagnostic criteria used and population under study. However, it remains one of the most underdiagnosed and undertreated conditions in ophthalmology. Many tests used in the diagnosis of DED rely on an experienced observer for image interpretation, which may be considered subjective and result in variation in diagnosis. Since artificial intelligence (AI) systems are capable of advanced problem solving, use of such techniques could lead to more objective diagnosis. Although the term `AI' is commonly used, recent succe…
Cross-Enhanced Multimodal Fusion of Eye-Tracking and Facial Features for Alzheimer's Disease Diagnosis
School of Control Science and Engineering, Shandong University(控制科学与工程学院,山东大学);Engineering Research Center of Intelligent Unmanned System, Ministry of Education(智能无人机系统工程研究中心,教育部);Department of Psychiatry, The Chinese University of Hong Kong(心理学系,香港中文大学);Department of Electronic Engineering, The Chinese University of Hong Kong(电子工程系,香港中文大学);AICARE Lab, Guangdong Medical University(AICARE实验室,广东医科大学);Department of Neurology, Shandong University of Traditional Chinese Medicine Affiliated Hospital(神经内科,山东中医药大学附属医院)
AI Workflow, External Validation, and Development in Eye Disease Diagnosis
National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆);Yale University(耶鲁大学);National Eye Institute, National Institutes of Health(美国国立卫生研究院国家眼科研究所);National University of Singapore(新加坡国立大学);Sydney University(悉尼大学);Oregon Health & Science University(俄勒冈健康与科学大学);VA Healthcare System(退伍军人医疗系统);Dartmouth Hitchcock Medical Center(达特茅斯-希区柯克医疗中心);Carolina Vision Center(卡罗来纳视觉中心);University of Miami(迈阿密大学);Krieger Eye Institute(克里格眼科研究所);University of California, Irvine(加利福尼亚大学尔湾分校);Fort Belvoir Community Hospital(贝尔沃堡社区医院);Uniformed Services University of the Health Sciences(健康科学统一服务大学);The George Washington University(乔治华盛顿大学);Singapore National Eye Center(新加坡国家眼科中心);Duke-NUS Medical School(杜克-新加坡国立大学医学院)
Enhancing Community Vision Screening -- AI Driven Retinal Photography for Early Disease Detection and Patient Trust
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局高性能计算研究所);Singapore Eye Research Institute(新加坡眼科研究所);Singapore National Eye Centre(新加坡国家眼科中心);Yong Loo Lin School of Medicine, NUS(新加坡国立大学杨潞龄医学院)
Fundus photography is indispensable for clinical detection and management of eye diseases. Limited image contrast and field of view (FOV) are common limitations of conventional fundus cameras, making it difficult to detect subtle abnormalities at the early stages of eye diseases. Further improvements of image contrast and FOV coverage are important to improve early disease detection and reliable treatment assessment. We report here a portable fundus camera, with a wide FOV and high dynamic range (HDR) imaging capabilities. Miniaturized indirect ophthalmoscopy illumination was employed to achie…
Purpose: To demonstrate that retinal microvasculature per se is a reliable biomarker for Diabetic Retinopathy (DR) and, by extension, cardiovascular diseases. Methods: Deep Learning Convolutional Neural Networks (CNN) applied to color fundus images for semantic segmentation of the blood vessels and severity classification on both vascular and full images. Vessel reconstruction through harmonic descriptors is also used as a smoothing and de-noising tool. The mathematical background of the theory is also outlined. Results: For diabetic patients, at least 93.8% of DR No-Refer vs. Refer classifica…
Diabetic Retinopathy (DR) is among the worlds leading vision loss causes in diabetic patients. DR is a microvascular disease that affects the eye retina, which causes vessel blockage and therefore cuts the main source of nutrition for the retina tissues. Treatment for this visual disorder is most effective when it is detected in its earliest stages, as severe DR can result in irreversible blindness. Nonetheless, DR identification requires the expertise of Ophthalmologists which is often expensive and time-consuming. Therefore, automatic detection systems were introduced aiming to facilitate th…
University of Liverpool(利物浦大学);Royal Liverpool University Hospital(皇家利物浦大学医院)
Prognostic models aim to predict the future course of a disease or condition and are a vital component of personalized medicine. Statistical models make use of longitudinal data to capture the temporal aspect of disease progression; however, these models require prior feature extraction. Deep learning avoids explicit feature extraction, meaning we can develop models for images where features are either unknown or impossible to quantify accurately. Previous prognostic models using deep learning with imaging data require annotation during training or only utilize a single time point. We propose …
Multiple Sclerosis (MS) and microvascular leukoencephalopathy are two distinct neurological conditions, the first caused by focal autoimmune inflammation in the central nervous system, the second caused by chronic white matter damage from atherosclerotic microvascular disease. Both conditions lead to signal anomalies on Fluid Attenuated Inversion Recovery (FLAIR) magnetic resonance (MR) images, which can be distinguished by an expert neuroradiologist, but which can look very similar to the untrained eye as well as in the early stage of both diseases. In this paper, we attempt to train a 3-dime…
Versatile Cataract Fundus Image Restoration Model Utilizing Unpaired Cataract and High-quality Images
Shenzhen-Platform and Carrier (International Science and Technology Information Center) & Shenzhen Bay Lab(深圳平台与载体(国际科技信息中心)及深圳湾实验室);Capital Medical University(首都医科大学);Beijing Tongren Hospital(北京同仁医院)
Adaptive Multiscale Retinal Diagnosis: A Hybrid Trio-Model Approach for Comprehensive Fundus Multi-Disease Detection Leveraging Transfer Learning and Siamese Networks
Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);Massachusetts Institute of Technology(麻省理工学院)
Optical coherence tomography (OCT) is a micrometer-scale, volumetric imaging modality that has become a clinical standard in ophthalmology. OCT instruments image by raster-scanning a focused light spot across the retina, acquiring sequential cross-sectional images to generate volumetric data. Patient eye motion during the acquisition poses unique challenges: Non-rigid, discontinuous distortions can occur, leading to gaps in data and distorted topographic measurements. We present a new distortion model and a corresponding fully-automatic, reference-free optimization strategy for computational m…