Retinal image quality assessment is an essential prerequisite for diagnosis of retinal diseases. Its goal is to identify retinal images in which anatomic structures and lesions attracting ophthalmologists' attention most are exhibited clearly and definitely while reject poor quality fundus images. Motivated by this, we mimic the way that ophthalmologists assess the quality of retinal images and propose a method termed SalStructuIQA. First, two salient structures for automated retinal quality assessment. One is the large-size salient structures including optic disc region and exudates in large-size. The other is the tiny-size salient structures which mainly include vessels. Then we incorporate the proposed two salient structure priors with deep convolutional neural network (CNN) to shift the focus of CNN to salient structures. Accordingly, we develop two CNN architectures: Dual-branch SalStructIQA and Single-branch SalStructIQA. Dual-branch SalStructIQA contains two CNN branches and one is guided by large-size salient structures while the other is guided by tiny-size salient structures. Single-branch SalStructIQA contains one CNN branch, which is guided by the concatenation of salient structures in both large-size and tiny-size. Experimental results on Eye-Quality dataset show that our proposed Dual-branch SalStructIQA outperforms the state-of-the-art methods for retinal image quality assessment and Single-branch SalStructIQA is much light-weight comparing with state-of-the-art deep retinal image quality assessment methods and still achieves competitive performances.
Central South University（中南大学）；Hunan Province Engineering Technology Research Center of Computer Vision and Intelligent Medical Treatment（湖南省计算机视觉与智能医疗工程技术研究中心）
Retinal image quality assessment is an essential task in the diagnosis of retinal diseases. Recently, there are emerging deep models to grade quality of retinal images. Current state-of-the-arts either directly transfer classification networks originally designed for natural images to quality classification of retinal images or introduce extra image quality priors via multiple CNN branches or independent CNNs. This paper proposes a dark and bright channel prior guided deep network for retinal image quality assessment called GuidedNet. Specifically, the dark and bright channel priors are embedd…
Inception Institute of Artificial Intelligence（起源人工智能研究院）；Baidu Inc.（百度公司）；Southern University of Science and Technology（南方科技大学）；Cixi Institute of Biomedical Engineering, CAS（中国科学院慈溪生物医学工程研究所）
Retinal image quality assessment (RIQA) is essential for controlling the quality of retinal imaging and guaranteeing the reliability of diagnoses by ophthalmologists or automated analysis systems. Existing RIQA methods focus on the RGB color-space and are developed based on small datasets with binary quality labels (i.e., `Accept' and `Reject'). In this paper, we first re-annotate an Eye-Quality (EyeQ) dataset with 28,792 retinal images from the EyePACS dataset, based on a three-level quality grading system (i.e., `Good', `Usable' and `Reject') for evaluating RIQA methods. Our RIQA dataset is …
University of Moratuwa（莫勒图沃大学）；Base Hospital Kebithigollawa（科比蒂戈勒沃基地医院）
The morphology of blood vessels in retinal fundus images is an important indicator of diseases like glaucoma, hypertension and diabetic retinopathy. The accuracy of retinal blood vessels segmentation affects the quality of retinal image analysis which is used in diagnosis methods in modern ophthalmology. Contrast enhancement is one of the crucial steps in any of retinal blood vessel segmentation approaches. The reliability of the segmentation depends on the consistency of the contrast over the image. This paper presents an assessment of the suitability of a recently invented spatially adaptive…
Contrastive Local Manifold Learning for No-Reference Image Quality Assessment
School of Information and Electronics, Beijing Institute of Technology（信息与电子学院，北京理工大学）；School of Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Xiamen University（多媒体可信感知与高效计算学院，厦门大学）；YouTu Laboratory, Tencent Company（YouTu实验室，腾讯公司）
Full end-to-end diagnostic workflow automation of 3D OCT via foundation model-driven AI for retinal diseases
Sun Yat-sen University（中山大学）；Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；The Hong Kong Polytechnic University（香港理工大学）；Zhongshan City People's Hospital（中山市人民医院）；Guangdong Medical University（广东医科大学）；The Third Affiliated Hospital of Sun Yat-sen University（中山大学附属第三医院）；Zhaoqing Gaoyao People’s Hospital（肇庆高要人民医院）；Tianjin Medical University Eye Hospital（天津医科大学眼科医院）；Capital Medical University（首都医科大学）；Beijing Tongren Hospital（北京同仁医院）
Australian e-Health Research Centre, CSIRO（澳大利亚电子健康研究中心，联邦科学与工业研究组织）；ACRV, The Australian National University（澳大利亚机器人视觉中心，澳大利亚国立大学）；University of California, Berkeley（加州大学伯克利分校）
Purpose To develop a computer based method for the automated assessment of image quality in the context of diabetic retinopathy (DR) to guide the photographer. Methods A deep learning framework was trained to grade the images automatically. A large representative set of 7000 color fundus images were used for the experiment which were obtained from the EyePACS that were made available by the California Healthcare Foundation. Three retinal image analysis experts were employed to categorize these images into Accept and Reject classes based on the precise definition of image quality in the context…
The intelligent video surveillance system (IVSS) can automatically analyze the content of the surveillance image (SI) and reduce the burden of the manual labour. However, the SIs may suffer quality degradations in the procedure of acquisition, compression, and transmission, which makes IVSS hard to understand the content of SIs. In this paper, we first conduct an example experiment (i.e. the face detection task) to demonstrate that the quality of the SIs has a crucial impact on the performance of the IVSS, and then propose a saliency-based deep neural network for the blind quality assessment o…
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；University College London（伦敦大学学院）
Image quality assessment (IQA) algorithm aims to quantify the human perception of image quality. Unfortunately, there is a performance drop when assessing the distortion images generated by generative adversarial network (GAN) with seemingly realistic texture. In this work, we conjecture that this maladaptation lies in the backbone of IQA models, where patch-level prediction methods use independent image patches as input to calculate their scores separately, but lack spatial relationship modeling among image patches. Therefore, we propose an Attention-based Hybrid Image Quality Assessment Netw…
Center for Signal and Information Processing（信号与信息处理中心）；School of Electrical and Computer Engineering（电气与计算机工程学院）；Georgia Institute of Technology（佐治亚理工学院）
This paper presents a full-reference image quality estimator based on color, structure, and visual system characteristics denoted as CSV. In contrast to the majority of existing methods, we quantify perceptual color degradations rather than absolute pixel-wise changes. We use the CIEDE2000 color difference formulation to quantify low-level color degradations and the Earth Mover's Distance between color name descriptors to measure significant color degradations. In addition to the perceptual color difference, CSV also contains structural and perceptual differences. Structural feature maps are o…
Retinal Layer Segmentation in OCT Images With 2.5D Cross-slice Feature Fusion Module for Glaucoma Assessment
Department of Mechanical Engineering, Yonsei University, Seoul, 03722, South Korea（延世大学机械工程系）；Institute of Vision Research, Department of Ophthalmology, Severance Hospital, Yonsei University, Seoul, 03722, South Korea（视觉研究所眼科部, 顺化医院, 延世大学, 首尔, 03722, 韩国）；Yonsei University College of Medicine, Seoul, 03722, South Korea（延世大学医学院）
National Technical University of Athens（雅典国立技术大学）；Instituto de Bioingenieria, Universidad Miguel Hernandez（米格尔·埃尔南德斯大学生物工程研究所）
Challenges in the field of retinal prostheses motivate the development of retinal models to accurately simulate Retinal Ganglion Cells (RGCs) responses. The goal of retinal prostheses is to enable blind individuals to solve complex, reallife visual tasks. In this paper, we introduce the functional assessment (FA) of retinal models, which describes the concept of evaluating the performance of retinal models on visual understanding tasks. We present a machine learning method for FA: we feed traditional machine learning classifiers with RGC responses generated by retinal models, to solve object a…
An average observer perceives the world in color instead of black and white. Moreover, the visual system focuses on structures and segments instead of individual pixels. Based on these observations, we propose a full reference objective image quality metric modeling visual system characteristics and chroma similarity in the perceptually uniform color domain (Lab). Laplacian of Gaussian features are obtained in the L channel to model the retinal ganglion cells in human visual system and color similarity is calculated over the a and b channels. In the proposed perceptual similarity index (PerSIM…
Optical coherence tomography (OCT) is commonly used to analyze retinal layers for assessment of ocular diseases. In this paper, we propose a method for retinal layer segmentation and quantification of uncertainty based on Bayesian deep learning. Our method not only performs end-to-end segmentation of retinal layers, but also gives the pixel wise uncertainty measure of the segmentation output. The generated uncertainty map can be used to identify erroneously segmented image regions which is useful in downstream analysis. We have validated our method on a dataset of 1487 images obtained from 15 …
Omnidirectional images, aka 360 images, can deliver immersive and interactive visual experiences. As their popularity has increased dramatically in recent years, evaluating the quality of 360 images has become a problem of interest since it provides insights for capturing, transmitting, and consuming this new media. However, directly adapting quality assessment methods proposed for standard natural images for omnidirectional data poses certain challenges. These models need to deal with very high-resolution data and implicit distortions due to the spherical form of the images. In this study, we…
Centre for Intelligent Machines, McGill University（麦吉尔大学智能机器中心）；McGill University（麦吉尔大学）
Following the major successes of self-attention and Transformers for image analysis, we investigate the use of such attention mechanisms in the context of Image Quality Assessment (IQA) and propose a novel full-reference IQA method, Vision Transformer for Attention Modulated Image Quality (VTAMIQ). Our method achieves competitive or state-of-the-art performance on the existing IQA datasets and significantly outperforms previous metrics in cross-database evaluations. Most patch-wise IQA methods treat each patch independently; this partially discards global information and limits the ability to …
Expert-Agnostic Ultrasound Image Quality Assessment using Deep Variational Clustering
All India Institute of Medical Sciences (AIIMS)（全印医学科学学院）；Purdue University (PU)（普渡大学）；Indian Institute of Technology (IIT)（印度理工学院）；Berea College（伯里亚学院）
MedIQA: A Scalable Foundation Model for Prompt-Driven Medical Image Quality Assessment
Faculty of Applied Sciences, Macao Polytechnic University, Macao, China（澳门理工学院）；Department of Engineer Science, University of Oxford, Oxford, UK（牛津大学工程科学系）；Great Bay University, Dongguan, China（东莞大湾大学）；College of Physics and Information Engineering, Fuzhou University, Fuzhou, China（福州大学物理与信息工程学院）；Shanghai Jiao Tong University, Shanghai, China（上海交通大学）；Department of Radiology, Shenzhen People’s Hospital, Shenzhen, China（深圳人民医院放射科）
Comprehensive Evaluation of OCT-based Automated Segmentation of Retinal Layer, Fluid and Hyper-Reflective Foci: Impact on Clinical Assessment of Diabetic Retinopathy Severity
School of Engineering Science, Simon Fraser University（西蒙弗雷泽大学工程科学学院）；School of Medicine, Wake Forest University（威克森林大学医学院）；Sankara Nethralaya（萨卡拉眼科医院）；Department of Ophthalmology and Visual Sciences, The University of British Columbia（不列颠哥伦比亚大学眼科与视觉科学系）；School of Biomedical Engineering, The University of British Columbia（不列颠哥伦比亚大学生物医学工程学院）；Institute of Ophthalmology, University College London（伦敦大学学院眼科研究所）；Department of Medical Physics and Biomedical Engineering, University College London（伦敦大学学院医学物理与生物医学工程系）；Department of Computer Science, Memorial University of Newfoundland（新斯科舍纪念大学计算机科学系）
University of Mons (UMONS)（蒙斯大学）；Faculty of Engineering (FPMs)（工程学院）
Since the early 2000s, computational visual saliency has been a very active research area. Each year, more and more new models are published in the main computer vision conferences. Nowadays, one of the big challenges is to find a way to fairly evaluate all of these models. In this paper, a new framework is proposed to assess models of visual saliency. This evaluation is divided into three experiments leading to the proposition of a new evaluation framework. Each experiment is based on a basic question: 1) there are two ground truths for saliency evaluation: what are the differences between ey…
Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Models
Hertie Institute for AI in Brain Health, University of Tübingen（图宾根大学赫蒂大脑健康人工智能研究所）；University of Tübingen（图宾根大学）；Tübingen AI Center（图宾根人工智能中心）；Center for Ophthalmology, University of Tübingen（图宾根大学眼科中心）；Institute of Ophthalmology, University College London（伦敦大学学院眼科研究所）；University College London（伦敦大学学院）
Nankai University（南开大学）；University of Central Florida（中佛罗里达大学）
We provide a comprehensive evaluation of salient object detection (SOD) models. Our analysis identifies a serious design bias of existing SOD datasets which assumes that each image contains at least one clearly outstanding salient object in low clutter. The design bias has led to a saturated high performance for state-of-the-art SOD models when evaluated on existing datasets. The models, however, still perform far from being satisfactory when applied to real-world daily scenes. Based on our analyses, we first identify 7 crucial aspects that a comprehensive and balanced dataset should fulfill. …
The Harker School（哈克学校）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）
Current analysis of tumor proliferation, the most salient prognostic biomarker for invasive breast cancer, is limited to subjective mitosis counting by pathologists in localized regions of tissue images. This study presents the first data-driven integrative approach to characterize the severity of tumor growth and spread on a categorical and molecular level, utilizing multiple biologically salient deep learning classifiers to develop a comprehensive prognostic model. Our approach achieves pathologist-level performance on three-class categorical tumor severity prediction. It additionally pionee…
Disentanglement and Assessment of Shortcuts in Ophthalmological Retinal Imaging Exams
Faculdade de Engenharia, Universidade do Porto, Portugal（葡萄牙波尔图大学工程学院）；Institute for Systems and Computer Engineering, Technology and Science, Porto, Portugal（葡萄牙波尔图系统与计算机工程研究所）；University of Oxford, United Kingdom（英国牛津大学）；Federal University of São Paulo, São Paulo, Brasil（巴西圣保罗联邦大学）
Kyung Hee University（庆熙大学）；University of Dhaka（达卡大学）
Objective image quality assessment (IQA) is imperative in the current multimedia-intensive world, in order to assess the visual quality of an image at close to a human level of ability. Many~parameters such as color intensity, structure, sharpness, contrast, presence of an object, etc., draw human attention to an image. Psychological vision research suggests that human vision is biased to the center area of an image and display screen. As a result, if the center part contains any visually salient information, it draws human attention even more and any distortion in that part will be better per…
Multi-parametric magnetic resonance imaging (mpMRI) has a growing role in detecting prostate cancer lesions. Thus, it is pertinent that medical professionals who interpret these scans reduce the risk of human error by using computer-aided detection systems. The variety of algorithms used in system implementation, however, has yielded mixed results. Here we investigate the best machine learning classifier for each prostate zone. We also discover salient features to clarify the models' classification rationale. Of the data provided, we gathered and augmented T2 weighted images and apparent diffu…
HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection
School of Artificial Intelligence, Jiangxi Normal University（江西师范大学人工智能学院）；Faculty of Business Information, Shanghai Business School（上海商学院商务信息学院）；School of Computer Science and Information Engineering, Shanghai Institute of Technology（上海应用技术大学计算机科学与信息工程学院）
Semi-Supervised Multi-Task Learning for Interpretable Quality As- sessment of Fundus Images
UNCPBA, CONICET, Yatiris Group, Pladema Institute（UNCPBA、CONICET、Yatiris集团、Pladema研究所）；Servicio de Oftalmología, Hospital de Alta Complejidad en Red ``El Cruce'' Dr. Néstor Carlos Kirchner（眼科服务，高级复杂性网络‘El Cruce’医院Dr. Néstor Carlos Kirchner）
DB-KAUNet: An Adaptive Dual Branch Kolmogorov-Arnold UNet for Retinal Vessel Segmentation
School of Computer Science and Software Engineering, Southwest University（西南大学计算机科学与软件工程学院）；Innovation Centre of Ministry of Education for Development and Diseases, the Sixth Affiliated Hospital, School of Medicine, South China University of Technology（华南理工大学医学院附属第六医院）；Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学 Yong Loo Lin 医学院创新与精准眼科健康中心）；Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学 Yong Loo Lin 医学院眼科部）；College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学院与生物信息工程学院）；School of Electrical Engineering Automation, Jiangxi University of Science and Technology（江西理工大学电气工程自动化学院）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）
One of the important parameters for the assessment of glaucoma is optic nerve head (ONH) evaluation, which usually involves depth estimation and subsequent optic disc and cup boundary extraction. Depth is usually obtained explicitly from imaging modalities like optical coherence tomography (OCT) and is very challenging to estimate depth from a single RGB image. To this end, we propose a novel method using adversarial network to predict depth map from a single image. The proposed depth estimation technique is trained and evaluated using individual retinal images from INSPIRE-stereo dataset. We …
Optical coherence tomography (OCT) is a powerful and noninvasive method for retinal imaging. In this paper, we introduce a fast segmentation method based on a new variant of spectral graph theory named diffusion maps. The research is performed on spectral domain (SD) OCT images depicting macular and optic nerve head appearance. The presented approach does not require edge-based image information and relies on regional image texture. Consequently, the proposed method demonstrates robustness in situations of low image contrast or poor layer-to-layer image gradients. Diffusion mapping is applied …
Visual attention information can be traced on cortical response but not on the retina: evidence from electrophysiological mouse data using natural images as stimuli
State Key Laboratory of VRTS, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；College of Computer Science and Technology, Qingdao University（青岛大学计算机科学与技术学院）；Stonybrook University（石溪大学）
Previous RGB-D salient object detection (SOD) methods have widely adopted deep learning tools to automatically strike a trade-off between RGB and D (depth), whose key rationale is to take full advantage of their complementary nature, aiming for a much-improved SOD performance than that of using either of them solely. However, such fully automatic fusions may not always be helpful for the SOD task because the D quality itself usually varies from scene to scene. It may easily lead to a suboptimal fusion result if the D quality is not considered beforehand. Moreover, as an objective factor, the D…
The College of New Jersey（新泽西学院）；University of Oxford（牛津大学）；Ludwig Institute for Cancer Research（路德维希癌症研究所）
Disease complications can alter vascular network morphology and disrupt tissue functioning. Diabetic retinopathy, for example, is a complication of types 1 and 2 diabetes mellitus that can cause blindness. Microvascular diseases are assessed by visual inspection of retinal images, but this can be challenging when diseases exhibit silent symptoms or patients cannot attend in-person meetings. We examine the performance of machine learning algorithms in detecting microvascular disease when trained on statistical and topological summaries of segmented retinal vascular images. We apply our methods …
Multimodal Deep Learning for Stroke Prediction and Detection using Retinal Imaging and Clinical Data
Division of Engineering, New York University Abu Dhabi（纽约大学阿布扎赫尔分校工程系）；Institute for Healthier Living Abu Dhabi（阿布扎赫尔健康生活研究所）；Eye Institute at Cleveland Clinic Abu Dhabi（阿布扎赫尔克利夫兰医学中心眼科研究所）；Canberra Hospital（堪培拉医院）
OMSN and FAROS: OCTA Microstructure Segmentation Network and Fully Annotated Retinal OCTA Segmentation Dataset
State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University（中山大学中山眼科中心眼科学国家重点实验室）；Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science（广东省眼科与视觉科学重点实验室）
Medical University Vienna（维也纳医科大学）；Department of Biomedical Imaging and Image-guided Therapy, Computational Imaging Research Lab, Medical University Vienna, Austria（维也纳医科大学生物医学成像与影像引导治疗系计算成像研究实验室）；Christian Doppler Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University Vienna, Austria（维也纳医科大学眼科与视光学系眼科图像分析基督教多普勒实验室）；Department of Ophthalmology and Optometry, Medical University Vienna, Austria（维也纳医科大学眼科与视光学系）
The automatic detection of disease related entities in retinal imaging data is relevant for disease- and treatment monitoring. It enables the quantitative assessment of large amounts of data and the corresponding study of disease characteristics. The presence of hyperreflective foci (HRF) is related to disease progression in various retinal diseases. Manual identification of HRF in spectral-domain optical coherence tomography (SD-OCT) scans is error-prone and tedious. We present a fully automated machine learning approach for segmenting HRF in SD-OCT scans. Evaluation on annotated OCT images o…
Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images
Ahsanullah University of Science and Technology（阿萨努拉科技大学）
National University of Sciences and Technology（国立科技大学）；Khalifa University（哈利法大学）
Objective: Glaucoma is the second leading cause of blindness worldwide. Glaucomatous progression can be easily monitored by analyzing the degeneration of retinal ganglion cells (RGCs). Many researchers have screened glaucoma by measuring cup-to-disc ratios from fundus and optical coherence tomography scans. However, this paper presents a novel strategy that pays attention to the RGC atrophy for screening glaucomatous pathologies and grading their severity. Methods: The proposed framework encompasses a hybrid convolutional network that extracts the retinal nerve fiber layer, ganglion cell with …
On Biases in a UK Biobank-based Retinal Image Classification Model
Big Data Institute, University of Oxford（牛津大学大数据研究所）；Department of Oncology, University of Oxford（牛津大学肿瘤学系）；Oxford University Hospitals NHS FT（牛津大学医院NHS基金会信托）
本文在 UK Biobank 视网膜图像上训练疾病分类模型，发现跨人群及评估中心的性能差异，并表明现有偏差缓解方法效果有限，需按偏差类型定制方案。
