Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretation
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）；Cornell University（康奈尔大学）；St Vincent’s Hospital Melbourne（墨尔本圣文森特医院）；Centre for Vision, Speech and Signal Processing（视觉、语音与信号处理中心）；University of Surrey（萨里大学）
Chong Wang, Fengbei Liu, Yuanhong Chen, Helen Frazer, Gustavo Carneiro
Recent advances in prototypical learning have shown remarkable potential to provide useful decision interpretations associating activation maps and predictions with class-specific training prototypes. Such prototypical learning has been well-studied for various single-label diseases, but for quite relevant and more challenging multi-label diagnosis, where multiple diseases are often concurrent within an image, existing prototypical learning models struggle to obtain meaningful activation maps and effective class prototypes due to the entanglement of the multiple diseases. In this paper, we present a novel Cross- and Intra-image Prototypical Learning (CIPL) framework, for accurate multi-label disease diagnosis and interpretation from medical images. CIPL takes advantage of common cross-image semantics to disentangle the multiple diseases when learning the prototypes, allowing a comprehensive understanding of complicated pathological lesions. Furthermore, we propose a new two-level alignment-based regularisation strategy that effectively leverages consistent intra-image information to enhance interpretation robustness and predictive performance. Extensive experiments show that our CIPL attains the state-of-the-art (SOTA) classification accuracy in two public multi-label benchmarks of disease diagnosis: thoracic radiography and fundus images. Quantitative interpretability results show that CIPL also has superiority in weakly-supervised thoracic disease localisation over other leading saliency- and prototype-based explanation methods.
Computer-aided diagnosis (CADx) systems have been shown to assist radiologists by providing classifications of all kinds of medical images like Computed tomography (CT) and Magnetic resonance (MR). Currently, convolutional neural networks play an important role in CADx. However, since CNN model should have a square-like input, it is usually difficult to directly apply the CNN algorithms on the irregular segmentation region of interests (ROIs) where the radiologists are interested in. In this paper, we propose a new approach to construct the model by extracting and converting the information of…
MAProtoNet: A Multi-scale Attentive Interpretable Prototypical Part Network for 3D Magnetic Resonance Imaging Brain Tumor Classification
Tokyo University of Agriculture and Technology (TUAT)（东京农工大学）；RIKEN Center for Advanced Intelligence Project (RIKEN AIP)（理化学研究所先进智能研究中心）；Graduate School of Medicine, Juntendo University（顺天堂大学医学院）；Faculty of Health Data Science, Juntendo University（顺天堂大学健康数据科学学部）；RIKEN（理化学研究所）
UiT The Arctic University of Norway（挪威北极大学）；Technical University of Berlin（柏林工业大学）
The recent trend of integrating multi-source Chest X-Ray datasets to improve automated diagnostics raises concerns that models learn to exploit source-specific correlations to improve performance by recognizing the source domain of an image rather than the medical pathology. We hypothesize that this effect is enforced by and leverages label-imbalance across the source domains, i.e, prevalence of a disease corresponding to a source. Therefore, in this work, we perform a thorough study of the effect of label-imbalance in multi-source training for the task of pneumonia detection on the widely use…
Slice-level Detection of Intracranial Hemorrhage on CT Using Deep Descriptors of Adjacent Slices
Vingroup Big Data Institute（Vingroup大数据研究院）；VinBigData JSC（VinBigData股份公司）；VinUni-Illinois Smart Health Center, College of Engineering & Computer Science（VinUni-伊利诺伊智慧健康中心，工程与计算机科学学院）
A Polarization and Radiomics Feature Fusion Network for the Classification of Hepatocellular Carcinoma and Intrahepatic Cholangiocarcinoma
University College London（伦敦大学学院）；Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Fujian Medical University Cancer Hospital（福建医科大学肿瘤医院）；Fujian Cancer Hospital（福建省肿瘤医院）
We consider the problem of image classification for the purpose of aiding doctors in dermatological diagnosis. Dermatological diagnosis poses two major challenges for standard off-the-shelf techniques: First, the data distribution is typically extremely long tailed. Second, intra-class variability is often large. To address the first issue, we formulate the problem as low-shot learning, where once deployed, a base classifier must rapidly generalize to diagnose novel conditions given very few labeled examples. To model diverse classes effectively, we propose Prototypical Clustering Networks (PC…
Glaucoma is a disease in which the optic nerve is chronically damaged by the elevation of the intra-ocular pressure, resulting in visual field defect. Therefore, it is important to monitor and treat suspected patients before they are confirmed with glaucoma. In this paper, we propose a 2-stage ranking-CNN that classifies fundus images as normal, suspicious, and glaucoma. Furthermore, we propose a method of using the class activation map as a mask filter and combining it with the original fundus image as an intermediate input. Our results have improved the average accuracy by about 10% over the…
Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València（瓦伦西亚理工大学生物工程研究与创新研究所）；Universidad Politécnica de Cartagena（卡塔赫纳理工大学）
Glaucoma is one of the leading causes of blindness worldwide and Optical Coherence Tomography (OCT) is the quintessential imaging technique for its detection. Unlike most of the state-of-the-art studies focused on glaucoma detection, in this paper, we propose, for the first time, a novel framework for glaucoma grading using raw circumpapillary B-scans. In particular, we set out a new OCT-based hybrid network which combines hand-driven and deep learning algorithms. An OCT-specific descriptor is proposed to extract hand-crafted features related to the retinal nerve fibre layer (RNFL). In paralle…
Third Affiliated Hospital of Zhengzhou University, China（郑州大学第三附属医院）；Lehigh University, USA（理海大学）
Objective: Ultrahigh-resolution optical coherence microscopy (OCM) has recently demonstrated its potential for accurate diagnosis of human cervical diseases. One major challenge for clinical adoption, however, is the steep learning curve clinicians need to overcome to interpret OCM images. Developing an intelligent technique for computer-aided diagnosis (CADx) to accurately interpret OCM images will facilitate clinical adoption of the technology and improve patient care. Methods: 497 high-resolution 3-D OCM volumes (600 cross-sectional images each) were collected from 159 ex vivo specimens of …
Unsupervised Causal Prototypical Networks for De-biased Interpretable Dermoscopy Diagnosis
Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems（浙江可及感知与智能系统重点实验室）；College of Computer Science and Technology（计算机科学与技术学院）；Hangzhou Dianzi University（杭州电子科技大学）；Xi'an Jiaotong University（西安交通大学）
The Chinese University of Hong Kong（香港中文大学）；Beihang University（北京航空航天大学）；The University of British Columbia（不列颠哥伦比亚大学）
Despite recent progress on semi-supervised federated learning (FL) for medical image diagnosis, the problem of imbalanced class distributions among unlabeled clients is still unsolved for real-world use. In this paper, we study a practical yet challenging problem of class imbalanced semi-supervised FL (imFed-Semi), which allows all clients to have only unlabeled data while the server just has a small amount of labeled data. This imFed-Semi problem is addressed by a novel dynamic bank learning scheme, which improves client training by exploiting class proportion information. This scheme consist…
Z-SSMNet: Zonal-aware Self-supervised Mesh Network for Prostate Cancer Detection and Diagnosis with Bi-parametric MRI
School of Computer Science, Faculty of Engineering, The University of Sydney（悉尼大学工程学院计算机科学学院）；College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）；Nepean Hospital（尼平医院）；Telehealth and Technology Centre, Nepean Blue Mountains Local Health District (NBMLHD)（尼平蓝山地方卫生区远程医疗与技术中心）
COVID-Net USPro: An Open-Source Explainable Few-Shot Deep Prototypical Network to Monitor and Detect COVID-19 Infection from Point-of-Care Ultrasound Images
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）；McGill University（麦吉尔大学）
Indian Institute of Technology Indore（印度理工学院印多尔分校）
Alzheimer's Disease (AD) is a neurodegenerative disease that is one of the significant causes of death in the elderly population. Many deep learning techniques have been proposed to diagnose AD using Magnetic Resonance Imaging (MRI) scans. Predicting AD using 2D slices extracted from 3D MRI scans is challenging as the inter-slice information gets lost. To this end, we propose a novel and lightweight framework termed 'Biceph-Net' for AD diagnosis using 2D MRI scans that model both the intra-slice and inter-slice information. Biceph-Net has been experimentally shown to perform similar to other S…
Accurate visualization of liver tumors and their surrounding blood vessels is essential for noninvasive diagnosis and prognosis prediction of tumors. In medical image segmentation, there is still a lack of in-depth research on the simultaneous segmentation of liver tumors and peritumoral blood vessels. To this end, we collect the first liver tumor, and vessel segmentation benchmark datasets containing 52 portal vein phase computed tomography images with liver, liver tumor, and vessel annotations. In this case, we propose a 3D U-shaped Cross-Attention Network (UCA-Net) that utilizes a tailored …
An Interpretable Cross-Attentive Multi-modal MRI Fusion Framework for Schizophrenia Diagnosis
Tulane University（杜兰大学）；Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)（跨机构神经成像与数据科学转化研究中心（TReNDS））；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
University of Catania（卡塔尼亚大学）；Northwestern University（西北大学）；Mayo Clinic（梅奥诊所）；University of Torino（都灵大学）
Early detection of precancerous cysts or neoplasms, i.e., Intraductal Papillary Mucosal Neoplasms (IPMN), in pancreas is a challenging and complex task, and it may lead to a more favourable outcome. Once detected, grading IPMNs accurately is also necessary, since low-risk IPMNs can be under surveillance program, while high-risk IPMNs have to be surgically resected before they turn into cancer. Current standards (Fukuoka and others) for IPMN classification show significant intra- and inter-operator variability, beside being error-prone, making a proper diagnosis unreliable. The established prog…
Multi-Scale Feature Fusion using Parallel-Attention Block for COVID-19 Chest X-ray Diagnosis
Rutgers Cancer Institute of New Jersey（罗格斯新泽西癌症研究所）；Rutgers Robert Wood Johnson Medical School（罗格斯罗伯特·伍德·约翰逊医学院）；The University of British Columbia（不列颠哥伦比亚大学）
Elsevier Science Publishers B.V.（爱思唯尔科学出版公司）；University of Winchester（温彻斯特大学）；Universidad de Granada（格拉纳达大学）；Universidad de Malaga（马拉加大学）；University of Leicester（莱斯特大学）
Respiratory diseases kill million of people each year. Diagnosis of these pathologies is a manual, time-consuming process that has inter and intra-observer variability, delaying diagnosis and treatment. The recent COVID-19 pandemic has demonstrated the need of developing systems to automatize the diagnosis of pneumonia, whilst Convolutional Neural Network (CNNs) have proved to be an excellent option for the automatic classification of medical images. However, given the need of providing a confidence classification in this context it is crucial to quantify the reliability of the model's predict…
Celiac Disease (CD) and Environmental Enteropathy (EE) are common causes of malnutrition and adversely impact normal childhood development. Both conditions require a tissue biopsy for diagnosis and a major challenge of interpreting clinical biopsy images to differentiate between these gastrointestinal diseases is striking histopathologic overlap between them. In the current study, we propose four diagnosis techniques for these diseases and address their limitations and advantages. First, the diagnosis between CD, EE, and Normal biopsies is considered, but the main challenge with this diagnosis…
We address the problem of few-shot semantic segmentation (FSS), which aims to segment novel class objects in a target image with a few annotated samples. Though recent advances have been made by incorporating prototype-based metric learning, existing methods still show limited performance under extreme intra-class object variations and semantically similar inter-class objects due to their poor feature representation. To tackle this problem, we propose a dual prototypical contrastive learning approach tailored to the FSS task to capture the representative semanticfeatures effectively. The main …
Deep neural networks for image-based screening and computer-aided diagnosis have achieved expert-level performance on various medical imaging modalities, including chest radiographs. Recently, several works have indicated that these state-of-the-art classifiers can be biased with respect to sensitive patient attributes, such as race or gender, leading to growing concerns about demographic disparities and discrimination resulting from algorithmic and model-based decision-making in healthcare. Fair machine learning has focused on mitigating such biases against disadvantaged or marginalised group…
Embracing Intra-Class Heterogeneity for Semi-Supervised Medical Image Segmentation: From Diversity to Precision
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Artificial Intelligence Institute, Shanghai University（上海大学人工智能研究院）；Department of Computer and Data Sciences, Case Western Reserve University（凯斯西储大学计算机与数据科学系）；Department of Biomedical Engineering, Case Western Reserve University（凯斯西储大学生物医学工程系）
University of Central Florida（中佛罗里达大学）；Foundation for Research and Technology Hellas（希腊研究与技术基金会）；Mayo Clinic（梅奥诊所）
Intraductal papillary mucinous neoplasm (IPMN) is a precursor to pancreatic ductal adenocarcinoma. While over half of patients are diagnosed with pancreatic cancer at a distant stage, patients who are diagnosed early enjoy a much higher 5-year survival rate of $34\%$ compared to $3\%$ in the former; hence, early diagnosis is key. Unique challenges in the medical imaging domain such as extremely limited annotated data sets and typically large 3D volumetric data have made it difficult for deep learning to secure a strong foothold. In this work, we construct two novel "inflated" deep network arch…
sMRI-PatchNet: A novel explainable patch-based deep learning network for Alzheimer's disease diagnosis and discriminative atrophy localisation with Structural MRI
Manchester Metropolitan University（曼彻斯特城市大学）；Brunel University（布鲁内尔大学）；University of Sheffield（谢菲尔德大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Histomorphology-Guided Prototypical Multi-Instance Learning for Breast Cancer WSI Classification
School of Biomedical Engineering, USTC（生物医学工程学院）；USTC Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE)（USTC医学影像、机器人、分析计算与学习中心）；Key Laboratory of Intelligent Information Processing, ICT, CAS（智能信息处理重点实验室）；State Key Laboratory of Precision and Intelligent Chemistry, USTC（精密与智能化学国家重点实验室）
Tencent Youtu Lab（腾讯优图实验室）；The Chinese University of Hong Kong（香港中文大学）
Unsupervised Domain Adaptation (UDA) aims to adapt the model trained on the labeled source domain to an unlabeled target domain. In this paper, we present Prototypical Contrast Adaptation (ProCA), a simple and efficient contrastive learning method for unsupervised domain adaptive semantic segmentation. Previous domain adaptation methods merely consider the alignment of the intra-class representational distributions across various domains, while the inter-class structural relationship is insufficiently explored, resulting in the aligned representations on the target domain might not be as easil…
Tecnologico de Monterrey（蒙特雷理工学院）；CHU Nancy（南锡大学中心医院）；Université de Lorraine（洛林大学）
Identifying the type of kidney stones can allow urologists to determine their formation cause, improving the early prescription of appropriate treatments to diminish future relapses. However, currently, the associated ex-vivo diagnosis (known as morpho-constitutional analysis, MCA) is time-consuming, expensive, and requires a great deal of experience, as it requires a visual analysis component that is highly operator dependant. Recently, machine learning methods have been developed for in-vivo endoscopic stone recognition. Shallow methods have been demonstrated to be reliable and interpretable…
Manchester Metropolitan University（曼彻斯特城市大学）；University of Sheffield（谢菲尔德大学）
Lesion diagnosis of skin lesions is a very challenging task due to high inter-class similarities and intra-class variations in terms of color, size, site and appearance among different skin lesions. With the emergence of computer vision especially deep learning algorithms, lesion diagnosis is made possible using these algorithms trained on dermoscopic images. Usually, deep classification networks are used for the lesion diagnosis to determine different types of skin lesions. In this work, we used pixel-wise classification network to provide lesion diagnosis rather than classification network. …
Intra-YOLO: A Small Object Detection Model for Caries and Molar-Incisor Hypomineralization in Intraoral Photography Based on Transfer Learning with Reinforcement Learning
Department of Mechanical Engineering, College of Engineering, National Yang Ming Chiao Tung University（国家阳明交通大学机械工程系）；Taipei Medical University Hospital（台北医学大学医院）；Wan Fang Hospital, Taipei Medical University（台北医学大学万芳医院）
IOSVLM: A 3D Vision-Language Model for Unified Dental Diagnosis from Intraoral Scans
ZJU-UIUC Institute, Zhejiang University, Haining, 314400, China（浙大浙ICU研究所，浙江大学，海宁，314400，中国）；Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine, Hangzhou, 310058, China（口腔医院，口腔医学院，浙江大学医学院，杭州，310058，中国）；Angelalign Research Institute, Angel Align Inc., Shanghai, 200011, China（天使对齐研究院，天使对齐公司，上海，200011，中国）
HoloDx: Knowledge- and Data-Driven Multimodal Diagnosis of Alzheimer's Disease
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Department of Neurology, Renji Hospital Affiliated to Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属仁济医院神经内科）
Cephalometric Landmark Detection across Ages with Prototypical Network
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Lingang Laboratory（临港实验室）；Shanghai Linkedcare Information Technology Co., Ltd.（上海联众医疗信息技术有限公司）；Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University（上海交通大学医学院附属第九人民医院）；Shanghai United Imaging Intelligence Co. Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
The University of Hong Kong（香港大学）；University of Chinese Academy of Sciences（中国科学院大学）
Locating lesions is important in the computer-aided diagnosis of X-ray images. However, box-level annotation is time-consuming and laborious. How to locate lesions accurately with few, or even without careful annotations is an urgent problem. Although several works have approached this problem with weakly-supervised methods, the performance needs to be improved. One obstacle is that general weakly-supervised methods have failed to consider the characteristics of X-ray images, such as the highly-structural attribute. We therefore propose the Cross-chest Graph (CCG), which improves the performan…
Technical University Munich（慕尼黑工业大学）；Ludwig Maximilians University（慕尼黑大学）
Alzheimer's disease (AD) has a complex and multifactorial etiology, which requires integrating information about neuroanatomy, genetics, and cerebrospinal fluid biomarkers for accurate diagnosis. Hence, recent deep learning approaches combined image and tabular information to improve diagnostic performance. However, the black-box nature of such neural networks is still a barrier for clinical applications, in which understanding the decision of a heterogeneous model is integral. We propose PANIC, a prototypical additive neural network for interpretable AD classification that integrates 3D image…
Self-aware and Cross-sample Prototypical Learning for Semi-supervised Medical Image Segmentation
Xidian University（西安电子科技大学）；The First Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第一附属医院）；AIQ；Warren Alpert Medical School of Brown University（布朗大学沃伦·阿尔珀特医学院）
Contrastive self-supervised learning (CSL) with a prototypical regularization has been introduced in learning meaningful representations for downstream tasks that require strong semantic information. However, to optimize CSL with a loss that performs the prototypical regularization aggressively, e.g., the ProtoNCE loss, might cause the "coagulation" of examples in the embedding space. That is, the intra-prototype diversity of samples collapses to trivial solutions for their prototype being well-separated from others. Motivated by previous works, we propose to mitigate this phenomenon by learni…
Prototypical Cross-domain Knowledge Transfer for Cervical Dysplasia Visual Inspection
National University of Singapore（新加坡国立大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；Northwestern Polytechnical University（西北工业大学）；Zhejiang Gongshang University（浙江工商大学）；Wuhan University（武汉大学）
Exploring the Challenge and Value of Deep Learning in Automated Skin Disease Diagnosis
Polytechnic Institute, Zhejiang University（浙江大学 polytechnic 院）；School of Social and Public Administration, East China University Of Science And Technology（东华大学社会科学与公共管理学院）；College of Science, National University of Defense Technology（国防科技大学科学学院）；School of Mathematical Sciences, Zhejiang University（浙江大学数学科学学院）
The identification of melanoma involves an integrated analysis of skin lesion images acquired using the clinical and dermoscopy modalities. Dermoscopic images provide a detailed view of the subsurface visual structures that supplement the macroscopic clinical images. Melanoma diagnosis is commonly based on the 7-point visual category checklist (7PC). The 7PC contains intrinsic relationships between categories that can aid classification, such as shared features, correlations, and the contributions of categories towards diagnosis. Manual classification is subjective and prone to intra- and inte…
University of New South Wales（新南威尔士大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）；Stanford University（斯坦福大学）
The current pandemic, caused by the outbreak of a novel coronavirus (COVID-19) in December 2019, has led to a global emergency that has significantly impacted economies, healthcare systems and personal wellbeing all around the world. Controlling the rapidly evolving disease requires highly sensitive and specific diagnostics. While real-time RT-PCR is the most commonly used, these can take up to 8 hours, and require significant effort from healthcare professionals. As such, there is a critical need for a quick and automatic diagnostic system. Diagnosis from chest CT images is a promising direct…
