Deep learning has seen tremendous interest in medical imaging, particularly in the use of convolutional neural networks (CNNs) for developing automated diagnostic tools. The facility of its non-invasive acquisition makes retinal fundus imaging amenable to such automated approaches. Recent work in analyzing fundus images using CNNs relies on access to massive data for training and validation - hundreds of thousands of images. However, data residency and data privacy restrictions stymie the applicability of this approach in medical settings where patient confidentiality is a mandate. Here, we showcase results for the performance of DL on small datasets to classify patient sex from fundus images - a trait thought not to be present or quantifiable in fundus images until recently. We fine-tune a Resnet-152 model whose last layer has been modified for binary classification. In several experiments, we assess performance in the small dataset context using one private (DOVS) and one public (ODIR) data source. Our models, developed using approximately 2500 fundus images, achieved test AUC scores of up to 0.72 (95% CI: [0.67, 0.77]). This corresponds to a mere 25% decrease in performance despite a nearly 1000-fold decrease in the dataset size compared to prior work in the literature. Even with a hard task like sex categorization from retinal images, we find that classification is possible with very small datasets. Additionally, we perform domain adaptation experiments between DOVS and ODIR; explore the effect of data curation on training and generalizability; and investigate model ensembling to maximize CNN classifier performance in the context of small development datasets.
Through anonymisation and accessibility, social media platforms have facilitated the proliferation of hate speech, prompting increased research in developing automatic methods to identify these texts. This paper explores the classification of sexism in text using a variety of deep neural network model architectures such as Long-Short-Term Memory (LSTMs) and Convolutional Neural Networks (CNNs). These networks are used in conjunction with transfer learning in the form of Bidirectional Encoder Representations from Transformers (BERT) and DistilBERT models, along with data augmentation, to perfor…
CSA-Graphs: A Privacy-Preserving Structural Dataset for Child Sexual Abuse Research
Universidade Estadual de Campinas (UNICAMP)（坎皮纳斯州立大学）；Instituto Federal de Educação, Ciência e Tecnologia de Minas Gerais (IFMG)（米纳斯吉拉斯联邦教育、科学和技术研究所）；Universidade de São Paulo (USP)（圣保罗大学）；Universidade Federal de Minas Gerais (UFMG)（米纳斯吉拉斯联邦大学）；Polícia Federal (PF)（联邦警察）；University of Sheffield（谢菲尔德大学）
Purpose: Convolutional neural networks can be trained to detect various conditions or patient traits based on retinal fundus photographs, some of which, such as the patient sex, are invisible to the expert human eye. Here we propose a methodology for explainable classification of fundus images to uncover the mechanism(s) by which CNNs successfully predict the labels. We used patient sex as a case study to validate our proposed methodology. Approach: First, we used a set of 4746 fundus images, including training, validation and test partitions, to fine-tune a pre-trained CNN on the sex classifi…
KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
Artificial Intelligence Department University of Isfahan（人工智能系伊斯法罕大学）；Labbafinejad Hospital Shahid Beheshti University of Medical Science（拉巴法尼贾医院谢赫·贝赫什提医学科学大学）；Department of Mechanical Engineering Isfahan University of Technology（机械工程系伊斯法罕技术大学）
Improve Retinal Artery/Vein Classification via Channel Couplin
Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing, China（北京大学医学部医学技术研究所）；Department of Biomedical Engineering, Peking University, Beijing, China（北京大学生物医学工程系）；National Biomedical Imaging Center, Peking University, Beijing, China（北京大学国家生物医学成像中心）；Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology（佐治亚理工学院生物医学工程系）；Emory University, Atlanta, GA, USA（埃默里大学）
Purpose: To create and evaluate the accuracy of an artificial intelligence Deep learning platform (ORAiCLE) capable of using only retinal fundus images to predict both an individuals overall 5 year cardiovascular risk (CVD) and the relative contribution of the component risk factors that comprise this risk. Methods: We used 165,907 retinal images from a database of 47,236 patient visits. Initially, each image was paired with biometric data age, ethnicity, sex, presence and duration of diabetes a HDL/LDL ratios as well as any CVD event wtihin 5 years of the retinal image acquisition. A risk sco…
Disentanglement and Assessment of Shortcuts in Ophthalmological Retinal Imaging Exams
Faculdade de Engenharia, Universidade do Porto, Portugal（葡萄牙波尔图大学工程学院）；Institute for Systems and Computer Engineering, Technology and Science, Porto, Portugal（葡萄牙波尔图系统与计算机工程研究所）；University of Oxford, United Kingdom（英国牛津大学）；Federal University of São Paulo, São Paulo, Brasil（巴西圣保罗联邦大学）
Most computer aided pathology detection systems rely on large volumes of quality annotated data to aid diagnostics and follow up procedures. However, quality assuring large volumes of annotated medical image data can be subjective and expensive. In this work we present a novel standardization framework that implements three few-shot learning (FSL) models that can be iteratively trained by atmost 5 images per 3D stack to generate multiple regional proposals (RPs) per test image. These FSL models include a novel parallel echo state network (ParESN) framework and an augmented U-net model. Additio…
Recent contrastive learning methods achieved state-of-the-art in low label regimes. However, the training requires large batch sizes and heavy augmentations to create multiple views of an image. With non-contrastive methods, the negatives are implicitly incorporated in the loss, allowing different images and modalities as pairs. Although the meta-information (i.e., age, sex) in medical imaging is abundant, the annotations are noisy and prone to class imbalance. In this work, we exploited already existing temporal information (different visits from a patient) in a longitudinal optical coherence…
Beihang University（北京航空航天大学）；National Institute of Informatics（国立情报学研究所）；Peng Cheng Laboratory（鹏城实验室）；Central South University（中南大学）
Though deep learning has shown successful performance in classifying the label and severity stage of certain disease, most of them give few evidence on how to make prediction. Here, we propose to exploit the interpretability of deep learning application in medical diagnosis. Inspired by Koch's Postulates, a well-known strategy in medical research to identify the property of pathogen, we define a pathological descriptor that can be extracted from the activated neurons of a diabetic retinopathy detector. To visualize the symptom and feature encoded in this descriptor, we propose a GAN based meth…
University of Agder (UiA)（阿格德大学）；Institute of Marine Research (IMR)（海洋研究所）
A wide range of applications in marine ecology extensively uses underwater cameras. Still, to efficiently process the vast amount of data generated, we need to develop tools that can automatically detect and recognize species captured on film. Classifying fish species from videos and images in natural environments can be challenging because of noise and variation in illumination and the surrounding habitat. In this paper, we propose a two-step deep learning approach for the detection and classification of temperate fishes without pre-filtering. The first step is to detect each single fish in a…
Early Alzheimer's Disease Detection from Retinal OCT Images: A UK Biobank Study
Department of Computer Engineering Isik University Istanbul, Turkey（计算机工程系伊斯肯大学）；Department of Artificial Intelligence and Data Engineering（人工智能与数据工程系）；Data Engineering Istanbul Technical University Istanbul, Turkey（数据工程系伊斯坦布尔技术大学）
Sirindhorn International Institute of Technology, Thammasat University（泰国国立法政大学诗琳通国际技术学院）
Diabetic retinopathy is an eye-related pathology creating abnormalities and causing visual impairment, proper treatment of which requires identifying irregularities. This research uses a hemorrhage detection method and compares classification of conventional and deep features. Especially, method identifies hemorrhage connected with blood vessels or reside at retinal border and reported challenging. Initially, adaptive brightness adjustment and contrast enhancement rectify degraded images. Prospective locations of hemorrhages are estimated by a Gaussian matched filter, entropy thresholding, and…
According to the World Health Organization, 285 million people worldwide live with visual impairment. The most commonly used imaging technique for diagnosis in ophthalmology is optical coherence tomography (OCT). However, analysis of retinal OCT requires trained ophthalmologists and time, making a comprehensive early diagnosis unlikely. A recent study established a diagnostic tool based on convolutional neural networks (CNN), which was trained on a large database of retinal OCT images. The performance of the tool in classifying retinal conditions was on par to that of trained medical experts. …
Beihang University（北京航空航天大学）；Peng Cheng Laboratory（鹏城实验室）；RIKEN AIP（理化学研究所人工智能研究中心）；University of Tokyo（东京大学）；Ningbo Institute of Industrial Technology, CAS（中国科学院宁波工业技术研究院）
Though deep learning has shown successful performance in classifying the label and severity stage of certain diseases, most of them give few explanations on how to make predictions. Inspired by Koch's Postulates, the foundation in evidence-based medicine (EBM) to identify the pathogen, we propose to exploit the interpretability of deep learning application in medical diagnosis. By determining and isolating the neuron activation patterns on which diabetic retinopathy (DR) detector relies to make decisions, we demonstrate the direct relation between the isolated neuron activation and lesions for…
We present a method for automated segmentation of the vasculature in retinal images. The method produces segmentations by classifying each image pixel as vessel or non-vessel, based on the pixel's feature vector. Feature vectors are composed of the pixel's intensity and continuous two-dimensional Morlet wavelet transform responses taken at multiple scales. The Morlet wavelet is capable of tuning to specific frequencies, thus allowing noise filtering and vessel enhancement in a single step. We use a Bayesian classifier with class-conditional probability density functions (likelihoods) described…
Prediction of Alzheimer's Disease Risk Factors from Retinal Images via Deep Learning: Development and Validation of Biologically Relevant Morphological Associations in the UK Biobank
J. Crayton Pruitt Family Dept. of Biomedical Engineering, University of Florida（朱·克雷顿·普瑞特生物医学工程系，佛罗里达大学）；University of Florida Research Computing（佛罗里达大学研究计算中心）；Meta AI (FAIR)（Meta AI（FAIR））；School of Behavioral and Brain Sciences, University of Texas at Dallas（德克萨斯大学达拉斯分校行为与脑科学学院）；Dept. of Electrical and Computer Engineering, University of Florida（佛罗里达大学电气与计算机工程系）；Dept. of Computer and Information Science and Engineering, University of Florida（佛罗里达大学计算机与信息科学与工程系）；Center for Cognitive Aging and Memory, University of Florida（佛罗里达大学认知衰老与记忆中心）
Rajavithi Hospital（拉贾维蒂医院）；Google AI（谷歌人工智能）；Google（谷歌）；Sankara Nethralaya（尚卡拉·内特拉拉亚医院）；Vajira Hospital（瓦吉拉医院）；Eye and Laser Center（眼科与激光中心）；East Bay Retina Consultants（东湾视网膜顾问诊所）；University of California San Francisco（加州大学旧金山分校）；Retina-Vitreous Associates Medical Group（视网膜玻璃体联合医疗集团）；University of Southern California（南加州大学）；Lamphun Hospital（南奔医院）；Somdejphrajaotaksin Maharaj Hospital（颂德帕昭达信玛哈拉医院）；Sawanpracharak Hospital（沙旺帕乍拉医院）；Nakhon Nayok Hospital（那空那育医院）；Photharam Hospital（波塔兰医院）；Prapokklao Hospital（巴博告医院）；Mahasarakham Hospital（马哈沙拉堪医院）；Nongbualamphu Hospital（廊磨喃蒲医院）；Pakchongnana Hospital（北冲纳纳医院）；Mukdahan Hospital（穆达汉医院）；Suratthani Hospital（素叻他尼医院）；Sungaikolok Hospital（双溪哥乐医院）；Bangkok Metropolitan Administration Public Health Center 7（曼谷市行政公署第七公共卫生中心）
Deep learning algorithms have been used to detect diabetic retinopathy (DR) with specialist-level accuracy. This study aims to validate one such algorithm on a large-scale clinical population, and compare the algorithm performance with that of human graders. 25,326 gradable retinal images of patients with diabetes from the community-based, nation-wide screening program of DR in Thailand were analyzed for DR severity and referable diabetic macular edema (DME). Grades adjudicated by a panel of international retinal specialists served as the reference standard. Across different severity levels of…
Improvement Strategies for Few-Shot Learning in OCT Image Classification of Rare Retinal Diseases
Department of Electrical Engineering, Yuan Ze University（电气工程系，元智大学）；Department of Electrical Engineering, National Taiwan Normal University（电气工程系，国立台湾师范大学）；Department of Ophthalmology, Far Eastern Memorial Hospital（眼科部，东部医疗院）
University of California, Berkeley（加利福尼亚大学伯克利分校）；International Computer Science Institute（国际计算机科学研究所）；University of California, Davis（加利福尼亚大学戴维斯分校）
Many diseases are classified based on human-defined rubrics that are prone to bias. Supervised neural networks can automate the grading of retinal fundus images, but require labor-intensive annotations and are restricted to the specific trained task. Here, we employed an unsupervised network with Non-Parametric Instance Discrimination (NPID) to grade age-related macular degeneration (AMD) severity using fundus photographs from the Age-Related Eye Disease Study (AREDS). Our unsupervised algorithm demonstrated versatility across different AMD classification schemes without retraining, and achiev…
Interpretable Few-Shot Retinal Disease Diagnosis with Concept-Guided Prompting of Vision-Language Models
Monash University（莫纳什大学）；Centre for Eye Research Australia（澳大利亚眼科研究中心）；Royal Victorian Eye and Ear Hospital（皇家维多利亚眼耳医院）；The University of Melbourne（墨尔本大学）；The Hong Kong Polytechnic University（香港理工大学）；Centre for Eye and Vision Research（眼与视觉研究中心）；Indian Institute of Technology Bombay（印度理工学院孟买分校）；Airdoc-Monash Research Lab（Airdoc-莫纳什研究实验室）
Robert Bosch GmbH（罗伯特·博世有限公司）；TU Dortmund University（多特蒙德工业大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Convolutional neural networks (CNNs) show impressive performance for image classification and detection, extending heavily to the medical image domain. Nevertheless, medical experts are sceptical in these predictions as the nonlinear multilayer structure resulting in a classification outcome is not directly graspable. Recently, approaches have been shown which help the user to understand the discriminative regions within an image which are decisive for the CNN to conclude to a certain class. Although these approaches could help to build trust in the CNNs predictions, they are only slightly sho…
Machine Learning Prediction of Cardiovascular Risk in Type 1 Diabetes Mellitus Using Radiomics Features from Multimodal Retinal Images
Universitat Politècnica de Catalunya (UPC)（加泰罗尼亚理工大学）；Institut Clínic d´Oftalmología (ICOF)（眼科临床研究所）；Hospital Clínic de Barcelona（巴塞罗那临床医院）；Intelligent Data Science and Artificial Intelligence (IDEAI-UPC) Research Center（智能数据科学与人工智能研究中心）；August Pi i Sunyer Biomedical Research Institute (IDIBAPS)（奥古斯特·皮伊·苏涅尔生物医学研究所）；School of Medicine, Universitat de Barcelona（巴塞罗那大学医学院）
Peking University（北京大学）；Beijing Institute of Big Data Research（北京大数据研究院）；Peking University First Hospital（北京大学第一医院）；Beijing Tongren Hospital, Capital Medical University（首都医科大学北京同仁医院）；Beijing Ophthalmology and Visual Science Key Laboratory（北京眼科学与视觉科学重点实验室）
The change of retinal vasculature is an early sign of many vascular and systematic diseases, such as diabetes and hypertension. Different behaviors of retinal arterioles and venules form an important metric to measure the disease severity. Therefore, an accurate classification of arterioles and venules is of great necessity. In this work, we propose a novel architecture of deep convolutional neural network for segmenting and classifying arterioles and venules on retinal fundus images. This network takes the original color fundus image as inputs and multi-class labels as outputs. We adopt the e…
Simultaneous column-based deep learning progression analysis of atrophy associated with AMD in longitudinal OCT studies
School of Computer Science and Engineering, The Hebrew University of Jerusalem（耶路撒冷希伯来大学计算机科学与工程学院）；The Alexander Grass Center for Bioengineering, The Hebrew University of Jerusalem（耶路撒冷希伯来大学亚历山大·格拉斯生物工程中心）；Hadassah Medical Center（哈达萨医疗中心）
University of Münster（明斯特大学）；Agaplesion Diakonieklinikum（阿加普莱西翁迪亚科尼医院）；University of Marburg（马尔堡大学）；The University of Melbourne（墨尔本大学）
Transgender individuals (TIs) show brain structural alterations that differ from their biological sex as well as their perceived gender. To substantiate evidence that the brain structure of TIs differs from male and female, we use a combined multivariate and univariate approach. Gray matter segments resulting from voxel-based morphometry preprocessing of $N = 1753$ cisgender (CG) healthy participants were used to train ($N=1402$) and validate (20 % hold-out; $N = 351$) a support-vector machine classifying the biological sex. As a second validation, we classified $N = 1104$ patients with depres…
Southern University of Science and Technology（南方科技大学）；Sun Yat-Sen University（中山大学）；Hunan University of Arts and Science（湖南文理学院）
Optical coherence tomography angiography (OCTA) is a novel non-invasive imaging technique that allows visualizations of vasculature and foveal avascular zone (FAZ) across retinal layers. Clinical researches suggest that the morphology and contour irregularity of FAZ are important biomarkers of various ocular pathologies. Therefore, precise segmentation of FAZ has great clinical interest. Also, there is no existing research reporting that FAZ features can improve the performance of deep diagnostic classification networks. In this paper, we propose a novel multi-level boundary shape and distance…
DeepRepViz: Identifying Confounders in Deep Learning Model Predictions
Humboldt-Universität zu Berlin（柏林洪堡大学）；Charité – Universitätsmedizin Berlin（柏林夏里特医学院）；Einstein Center for Neurosciences Berlin（柏林爱因斯坦神经科学中心）；Freie Universität Berlin（柏林自由大学）
Beyond Off-the-Shelf Models: A Lightweight and Accessible Machine Learning Pipeline for Ecologists Working with Image Data
Department of Statistics, LMU Munich（统计学系，慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Research Unit Wildlife Biology and Management, Bavarian State Institute for Forestry（林业野生动物生物学与管理研究组，巴伐利亚州林业研究所）
Technical University of Munich（慕尼黑工业大学）；Heritage Institute of Technology（印度传统理工学院）
Automatic evaluation of the retinal fundus image is emerging as one of the most important tools for early detection and treatment of progressive eye diseases like Glaucoma. Glaucoma results to a progressive degeneration of vision and is characterized by the deformation of the shape of optic cup and the degeneration of the blood vessels resulting in the formation of a notch along the neuroretinal rim. In this paper, we propose a deep learning-based pipeline for automatic segmentation of optic disc (OD) and optic cup (OC) regions from Digital Fundus Images (DFIs), thereby extracting distinct fea…
Carmel High School（卡梅尔高中）；Capstone School Dhaka（达卡顶点学校）；Clarkstown High School South（克拉克镇南部高中）；University at Albany, State University of New York（纽约州立大学奥尔巴尼分校）；Purdue University（普渡大学）
National Center for Biotechnology Information (NCBI)（美国国家生物技术信息中心）；National Library of Medicine (NLM)（美国国家医学图书馆）；National Institutes of Health (NIH)（美国国立卫生研究院）；National Eye Institute (NEI)（美国国家眼科研究所）
In assessing the severity of age-related macular degeneration (AMD), the Age-Related Eye Disease Study (AREDS) Simplified Severity Scale predicts the risk of progression to late AMD. However, its manual use requires the time-consuming participation of expert practitioners. Although several automated deep learning systems have been developed for classifying color fundus photographs (CFP) of individual eyes by AREDS severity score, none to date has used a patient-based scoring system that uses images from both eyes to assign a severity score. DeepSeeNet, a deep learning model, was developed to c…
