We consider the problem of classifying a medical image dataset when we have a limited amounts of labels. This is very common yet challenging setting as labelled data is expensive, time consuming to collect and may require expert knowledge. The current classification go-to of deep supervised learning is unable to cope with such a problem setup. However, using semi-supervised learning, one can produce accurate classifications using a significantly reduced amount of labelled data. Therefore, semi-supervised learning is perfectly suited for medical image classification. However, there has almost been no uptake of semi-supervised methods in the medical domain. In this work, we propose an all-in-one framework for deep semi-supervised classification focusing on graph based approaches, which up to our knowledge it is the first time that an approach with minimal labels has been shown to such an unprecedented scale with medical data. We introduce the concept of hybrid models by defining a classifier as a combination between an energy-based model and a deep net. Our energy functional is built on the Dirichlet energy based on the graph p-Laplacian. Our framework includes energies based on the $\ell_1$ and $\ell_2$ norms. We then connected this energy model to a deep net to generate a much richer feature space to construct a stronger graph. Our framework can be set to be adapted to any complex dataset. We demonstrate, through extensive numerical comparisons, that our approach readily compete with fully-supervised state-of-the-art techniques for the applications of Malaria Cells, Mammograms and Chest X-ray classification whilst using only 20% of labels.
Union Strong Technology Beijing Co.（北京联强科技公司）；The University of Virginia（弗吉尼亚大学）；Beijing Tiantan hospital（北京天坛医院）
Intracranial aneurysm (IA) is a life-threatening blood spot in human's brain if it ruptures and causes cerebral hemorrhage. It is challenging to detect whether an IA has ruptured from medical images. In this paper, we propose a novel graph based neural network named GraphNet to detect IA rupture from 3D surface data. GraphNet is based on graph convolution network (GCN) and is designed for graph-level classification and node-level segmentation. The network uses GCN blocks to extract surface local features and pools to global features. 1250 patient data including 385 ruptured and 865 unruptured …
In clinical practice, human radiologists actually review medical images with high resolution monitors and zoom into region of interests (ROIs) for a close-up examination. Inspired by this observation, we propose a hierarchical graph neural network to detect abnormal lesions from medical images by automatically zooming into ROIs. We focus on mammogram analysis for breast cancer diagnosis for this study. Our proposed network consist of two graph attention networks performing two tasks: (1) node classification to predict whether to zoom into next level; (2) graph classification to classify whethe…
SegVol: Universal and Interactive Volumetric Medical Image Segmentation
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；BAAI（北京智源人工智能研究院）；The Chinese University of Hong Kong（香港中文大学）；Peking University（北京大学）
Petuum, Inc.（Petuum公司）；Cornell University（康奈尔大学）；UiT The Arctic University of Norway（挪威北极大学）；Carnegie Mellon University（卡内基梅隆大学）
Convolutional neural networks have led to significant breakthroughs in the domain of medical image analysis. However, the task of breast cancer segmentation in whole-slide images (WSIs) is still underexplored. WSIs are large histopathological images with extremely high resolution. Constrained by the hardware and field of view, using high-magnification patches can slow down the inference process and using low-magnification patches can cause the loss of information. In this paper, we aim to achieve two seemingly conflicting goals for breast cancer segmentation: accurate and fast prediction. We p…
Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）；Singapore Eye Research Institute（新加坡眼科研究所）；Singapore National Eye Centre（新加坡全国眼科中心）；Yong Loo Lin School of Medicine, NUS（新加坡国立大学杨潞龄医学院）
Localizing anatomical landmarks are important tasks in medical image analysis. However, the landmarks to be localized often lack prominent visual features. Their locations are elusive and easily confused with the background, and thus precise localization highly depends on the context formed by their surrounding areas. In addition, the required precision is usually higher than segmentation and object detection tasks. Therefore, localization has its unique challenges different from segmentation or detection. In this paper, we propose a zoom-in attentive network (ZIAN) for anatomical landmark loc…
Auckland Bioengineering Institute, The University of Auckland（奥克兰大学生物工程研究所）
Size of the training dataset is an important factor in the performance of a machine learning algorithms and tools used in medical image processing are not exceptions. Machine learning tools normally require a decent amount of training data before they could efficiently predict a target. For image processing and computer vision, the number of images determines the validity and reliability of the training set. Medical images in some cases, suffer from poor quality and inadequate quantity required for a suitable training set. The proposed algorithm in this research obviates the need for large or …
Cell detection is a fundamental task in computational pathology that can be used for extracting high-level medical information from whole-slide images. For accurate cell detection, pathologists often zoom out to understand the tissue-level structures and zoom in to classify cells based on their morphology and the surrounding context. However, there is a lack of efforts to reflect such behaviors by pathologists in the cell detection models, mainly due to the lack of datasets containing both cell and tissue annotations with overlapping regions. To overcome this limitation, we propose and publicl…
Segmentation of ultra-high resolution images is increasingly demanded, yet poses significant challenges for algorithm efficiency, in particular considering the (GPU) memory limits. Current approaches either downsample an ultra-high resolution image or crop it into small patches for separate processing. In either way, the loss of local fine details or global contextual information results in limited segmentation accuracy. We propose collaborative Global-Local Networks (GLNet) to effectively preserve both global and local information in a highly memory-efficient manner. GLNet is composed of a gl…
MedReason-R1: Learning to Reason for CT Diagnosis with Reinforcement Learning and Local Zoom
1 School of Biomedical Engineering, Division of Life Sciences；Medicine, University of Science；Technology of China, Hefei, Anhui, 230026, P.R. China 2 Center for Medical Imaging, Robotics；Analytic Computing \& LEarning (MIRACLE), Suzhou Institute for Advanced Research, USTC, Suzhou 215123, P.R. China 3 Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, Suzhou Jiangsu, 215123 4 State Key Laboratory of Precision
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）
Automatic organ segmentation is an important yet challenging problem for medical image analysis. The pancreas is an abdominal organ with very high anatomical variability. This inhibits previous segmentation methods from achieving high accuracies, especially compared to other organs such as the liver, heart or kidneys. In this paper, we present a probabilistic bottom-up approach for pancreas segmentation in abdominal computed tomography (CT) scans, using multi-level deep convolutional networks (ConvNets). We propose and evaluate several variations of deep ConvNets in the context of hierarchical…
College of Biomedical Engineering, Rangsit University（兰实大学生物医学工程学院）
Automated medical image segmentation is a priority research area for computational methods. In particular, detection of cancerous tumors represents a current challenge in this area with potential for real-world impact. This paper describes a method developed in response to the 2019 Kidney Tumor Segmentation Challenge (KiTS19). Axial computed tomography (CT) scans from 210 kidney cancer patients were used to develop and evaluate this automatic segmentation method based on a logical ensemble of fully-convolutional network (FCN) architectures, followed by volumetric validation. Data was pre-proce…
An increasing number of applications in computer vision, specially, in medical imaging and remote sensing, become challenging when the goal is to classify very large images with tiny informative objects. Specifically, these classification tasks face two key challenges: $i$) the size of the input image is usually in the order of mega- or giga-pixels, however, existing deep architectures do not easily operate on such big images due to memory constraints, consequently, we seek a memory-efficient method to process these images; and $ii$) only a very small fraction of the input images are informati…
Information and Computer Science, King Fahd University of Petroleum and Minerals（信息与计算机科学系，国王法赫德石油与矿物大学）；Faculty of Science and Information Technology, Daffodil International University（科学与信息技术学院，达福尔国际大学）；Department of Human Anatomy and Physiology, The Faculty of Health Sciences, University of Johannesburg（人类解剖学与生理学系，健康科学学院，约翰内斯堡大学）；Centre for Data Analytics（数据分析中心）；School of Psychology, Bond University（心理学学院，邦德大学）
Context: Mpox is a zoonotic disease caused by the Mpox virus, which shares similarities with other skin conditions, making accurate early diagnosis challenging. Artificial intelligence (AI), especially Deep Learning (DL), has a strong tool for medical image analysis; however, pre-trained models like CNNs and XAI techniques for mpox detection is underexplored. Objective: This study aims to evaluate the effectiveness of pre-trained CNN models (VGG16, VGG19, InceptionV3, MobileNetV2) for the early detection of monkeypox using binary and multi-class datasets. It also seeks to enhance model interpr…
WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis
College of Artificial Intelligence, Shenzhen University, China（深圳大学人工智能学院）；School of Computer Science, University of Nottingham Ningbo China, China（诺丁汉宁波大学计算机学院）；College of Computer Science and Software Engineering, Shenzhen University, China（深圳大学计算机科学与软件工程学院）；Department of Electrical Engineering, City University of Hong Kong, Hong Kong（香港城市大学电气工程系）；Wuyi University, China（武夷大学）
Blackbox Adaptation for Medical Image Segmentation
The Johns Hopkins University（约翰斯·霍普金斯大学）；Wilmer Eye Institute, The Johns Hopkins University（约翰斯·霍普金斯大学威尔眼科研究所）；Malone Center for Engineering in Healthcare, The Johns Hopkins University（约翰斯·霍普金斯大学马龙医疗保健工程中心）；Department of Electrical and Computer Engineering, The Johns Hopkins University（约翰斯·霍普金斯大学电气与计算机工程系）
Recent breakthroughs in object detection and image classification using Convolutional Neural Networks (CNNs) are revolutionizing the state of the art in medical imaging, and microscopy in particular presents abundant opportunities for computer vision algorithms to assist medical professionals in diagnosis of diseases ranging from malaria to cancer. High resolution scans of microscopy slides called Whole Slide Images (WSIs) offer enough information for a cancer pathologist to come to a conclusion regarding cancer presence, subtype, and severity based on measurements of features within the slide…
