Capsule network is a recent new deep network architecture that has been applied successfully for medical image segmentation tasks. This work extends capsule networks for volumetric medical image segmentation with self-supervised learning. To improve on the problem of weight initialization compared to previous capsule networks, we leverage self-supervised learning for capsule networks pre-training, where our pretext-task is optimized by self-reconstruction. Our capsule network, SS-3DCapsNet, has a UNet-based architecture with a 3D Capsule encoder and 3D CNNs decoder. Our experiments on multiple datasets including iSeg-2017, Hippocampus, and Cardiac demonstrate that our 3D capsule network with self-supervised pre-training considerably outperforms previous capsule networks and 3D-UNets.
Convolutional Neural Networks (CNNs) have achieved promising results in medical image segmentation. However, CNNs require lots of training data and are incapable of handling pose and deformation of objects. Furthermore, their pooling layers tend to discard important information such as positions as well as CNNs are sensitive to rotation and affine transformation. Capsule network is a recent new architecture that has achieved better robustness in part-whole representation learning by replacing pooling layers with dynamic routing and convolutional strides, which has shown potential results on po…
VinAI Research（VinAI研究院）；University of Arkansas（阿肯色大学）
Medical image segmentation has been so far achieving promising results with Convolutional Neural Networks (CNNs). However, it is arguable that in traditional CNNs, its pooling layer tends to discard important information such as positions. Moreover, CNNs are sensitive to rotation and affine transformation. Capsule network is a data-efficient network design proposed to overcome such limitations by replacing pooling layers with dynamic routing and convolutional strides, which aims to preserve the part-whole relationships. Capsule network has shown a great performance in image recognition and nat…
Retinal Vessel Segmentation with Deep Graph and Capsule Reasoning
Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China（电子科技大学长三角研究院（湖州））；School of Life Science and Technology, University of Electronic Science and Technology of China（电子科技大学生命科学与技术学院）；University of South Florida（南佛罗里达大学）
Eindhoven University of Technology（埃因霍温理工大学）；Philips GmbH Innovative Technologies（飞利浦创新技术有限公司）
Automatic segmentation of anatomical structures with convolutional neural networks (CNNs) constitutes a large portion of research in medical image analysis. The majority of CNN-based methods rely on an abundance of labeled data for proper training. Labeled medical data is often scarce, but unlabeled data is more widely available. This necessitates approaches that go beyond traditional supervised learning and leverage unlabeled data for segmentation tasks. This work investigates the potential of autoencoder-extracted features to improve segmentation with a CNN. Two strategies were considered. F…
Semi-supervised learning methods have been explored in medical image segmentation tasks due to the scarcity of pixel-level annotation in the real scenario. Proto-type alignment based consistency constraint is an intuitional and plausible solu-tion to explore the useful information in the unlabeled data. In this paper, we propose a mutual- and self- prototype alignment (MSPA) framework to better utilize the unlabeled data. In specific, mutual-prototype alignment enhances the information interaction between labeled and unlabeled data. The mutual-prototype alignment imposes two consistency constr…
Foundation Model-guided Iteratively Prompting and Pseudo-Labeling for Partially Labeled Medical Image Segmentation
Department of Biomedical Engineering, Columbia University（哥伦比亚大学生物医学工程系）；Department of Radiation Oncology, Columbia University Irving Medical Center（哥伦比亚大学欧文医学中心放射肿瘤科）；Data Science Institute, Columbia University（哥伦比亚大学数据科学研究所）；Herbert Irving Comprehensive Cancer Center, Columbia University（哥伦比亚大学赫伯特·欧文综合癌症中心）；Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院放射肿瘤科）
Convolutional Neural Networks (CNNs) have been successful in solving tasks in computer vision including medical image segmentation due to their ability to automatically extract features from unstructured data. However, CNNs are sensitive to rotation and affine transformation and their success relies on huge-scale labeled datasets capturing various input variations. This network paradigm has posed challenges at scale because acquiring annotated data for medical segmentation is expensive, and strict privacy regulations. Furthermore, visual representation learning with CNNs has its own flaws, e.g…
University of Augsburg（奥格斯堡大学）；University Hospital Augsburg（奥格斯堡大学医院）
Performance measures are an important tool for assessing and comparing different medical image segmentation algorithms. Unfortunately, the current measures have their weaknesses when it comes to assessing certain edge cases. These limitations arouse when images with a very small region of interest or without a region of interest at all are assessed. As a solution for these limitations, we propose a new medical image segmentation metric: MISm. To evaluate MISm, the popular metrics in the medical image segmentation and MISm were compared using images of magnet resonance tomography from several s…
Capsule-ConvKAN: A Hybrid Neural Approach to Medical Image Classification
Department of Cybernetics and Artificial Intelligence, Technical University of Košice（控制论与人工智能系，科希斯技术大学）；Institute of Information Science, University of Miskolc（信息科学研究所，米什科尔特大学）；Department of Computer Science and Engineering, University of Connecticut（计算机科学与工程系，康涅狄格大学）
Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation
Fuzhou University（福州大学）；Fujian Medical University Union Hospital（福建医科大学附属协和医院）；Fujian Medical Ultrasound Research Institute（福建省医学超声研究所）；Macao Polytechnic University（澳门理工大学）
Due to the limitation of available labeled data, medical image segmentation is a challenging task for deep learning. Traditional data augmentation techniques have been shown to improve segmentation network performances by optimizing the usage of few training examples. However, current augmentation approaches for segmentation do not tackle the strong texture bias of convolutional neural networks, observed in several studies. This work shows on the MoNuSeg dataset that style augmentation, which is already used in classification tasks, helps reducing texture over-fitting and improves segmentation…
Pairwise Similarity Regularization for Semi-supervised Graph Medical Image Segmentation
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机学院）；Academy of Military Sciences（军事科学院）；Peking University（北京大学）；Hebei University of Technology（河北工业大学）
Pseudo Label-Guided Data Fusion and Output Consistency for Semi-Supervised Medical Image Segmentation
college of physics and information engineering, University of Fuzhou University（福州大学物理与信息工程学院）；Faculty of Applied Science, University of Macao Polytechnic University（澳门理工大学应用科学学院）；University of Toronto University（多伦多大学）
Semi-Supervised Medical Image Segmentation via Dual Networks
AIPM, School of Artificial Intelligence, Guilin University of Electronic Technology（桂林电子科技大学人工智能学院AIPM）；College of Computer Science and Engineering, Jeddah University（吉达大学计算机科学与工程学院）
Xiamen University（厦门大学）；Chinese PLA General Hospital（中国人民解放军总医院）；Deepwise AI Laboratory（深度智耀人工智能实验室）
Standard segmentation of medical images based on full-supervised convolutional networks demands accurate dense annotations. Such learning framework is built on laborious manual annotation with restrict demands for expertise, leading to insufficient high-quality labels. To overcome such limitation and exploit massive weakly labeled data, we relaxed the rigid labeling requirement and developed a semi-supervised learning framework based on a teacher-student fashion for organ and lesion segmentation with partial dense-labeled supervision and supplementary loose bounding-box supervision which are e…
University of Central Florida（中佛罗里达大学）；University of Pennsylvania（宾夕法尼亚大学）
Convolutional neural network based systems have largely failed to be adopted in many high-risk application areas, including healthcare, military, security, transportation, finance, and legal, due to their highly uninterpretable "black-box" nature. Towards solving this deficiency, we teach a novel multi-task capsule network to improve the explainability of predictions by embodying the same high-level language used by human-experts. Our explainable capsule network, X-Caps, encodes high-level visual object attributes within the vectors of its capsules, then forms predictions based solely on these…
University of Notre Dame（圣母大学）；Guangdong Provincial People’s Hospital（广东省人民医院）
The success of deep learning methods in medical image segmentation tasks heavily depends on a large amount of labeled data to supervise the training. On the other hand, the annotation of biomedical images requires domain knowledge and can be laborious. Recently, contrastive learning has demonstrated great potential in learning latent representation of images even without any label. Existing works have explored its application to biomedical image segmentation where only a small portion of data is labeled, through a pre-training phase based on self-supervised contrastive learning without using a…
Partially Supervised Unpaired Multi-Modal Learning for Label-Efficient Medical Image Segmentation
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（高性能计算研究所（A*STAR））；The Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University（山东大学-新加坡南洋理工大学人工智能联合研究中心）
Recently, several Bayesian deep learning methods have been proposed for semi-supervised medical image segmentation. Although they have achieved promising results on medical benchmarks, some problems are still existing. Firstly, their overall architectures belong to the discriminative models, and hence, in the early stage of training, they only use labeled data for training, which might make them overfit to the labeled data. Secondly, in fact, they are only partially based on Bayesian deep learning, as their overall architectures are not designed under the Bayesian framework. However, unifying …
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Medicine Essen（埃森大学医学院）
Producing densely annotated data is a difficult and tedious task for medical imaging applications. To address this problem, we propose a novel approach to generate supervision for semi-supervised semantic segmentation. We argue that visually similar regions between labeled and unlabeled images likely contain the same semantics and therefore should share their label. Following this thought, we use a small number of labeled images as reference material and match pixels in an unlabeled image to the semantics of the best fitting pixel in a reference set. This way, we avoid pitfalls such as confirm…
GIGP: A Global Information Interacting and Geometric Priors Focusing Framework for Semi-supervised Medical Image Segmentation
The School of Mathematical Science, Capital Normal University（首都师范大学数学科学学院）；The Academy for Multidisciplinary Studies, Capital Normal University（首都师范大学交叉科学研究院）
Samsung Electronics（三星电子）；Seoul National University（首尔国立大学）；Soongsil University（崇实大学）
Semi-supervised learning for medical image segmentation is an important area of research for alleviating the huge cost associated with the construction of reliable large-scale annotations in the medical domain. Recent semi-supervised approaches have demonstrated promising results by employing consistency regularization, pseudo-labeling techniques, and adversarial learning. These methods primarily attempt to learn the distribution of labeled and unlabeled data by enforcing consistency in the predictions or embedding context. However, previous approaches have focused only on local discrepancy mi…
Style-Aware Blending and Prototype-Based Cross-Contrast Consistency for Semi-Supervised Medical Image Segmentation
School of Information Science and Engineering（信息科学与工程学院）；Yunnan University（云南大学）；Yunnan Key Laboratory of Intelligent Systems and Computing（智能系统与计算云南省重点实验室）
Dual Cross-image Semantic Consistency with Self-aware Pseudo Labeling for Semi-supervised Medical Image Segmentation
School of Biomedical Engineering & State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University（生物医学工程学院及先进医疗材料与器件国家重点实验室，上海科技大学）；Lingang Laboratory（临港实验室）；Department of Radiology, Stanford University（放射科，斯坦福大学）
Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation
The Chinese University of Hong Kong（香港中文大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Institute of Medical Intelligence and XR, The Chinese University of Hong Kong（香港中文大学医学智能与XR研究所）
提出 CPC-SAM，在双分支框架中通过 SAM 交叉提示和提示一致性正则化利用标注与未标注数据，显著提升半监督医学图像分割性能。
Northwestern Polytechnical University（西北工业大学）；The University of Adelaide（阿德莱德大学）
Due to the intensive cost of labor and expertise in annotating 3D medical images at a voxel level, most benchmark datasets are equipped with the annotations of only one type of organs and/or tumors, resulting in the so-called partially labeling issue. To address this, we propose a dynamic on-demand network (DoDNet) that learns to segment multiple organs and tumors on partially labeled datasets. DoDNet consists of a shared encoder-decoder architecture, a task encoding module, a controller for generating dynamic convolution filters, and a single but dynamic segmentation head. The information of …
Recent trends in semi-supervised learning have significantly boosted the performance of 3D semi-supervised medical image segmentation. Compared with 2D images, 3D medical volumes involve information from different directions, e.g., transverse, sagittal, and coronal planes, so as to naturally provide complementary views. These complementary views and the intrinsic similarity among adjacent 3D slices inspire us to develop a novel annotation way and its corresponding semi-supervised model for effective segmentation. Specifically, we firstly propose the orthogonal annotation by only labeling two o…
Northeastern University（东北大学）；The University of Auckland（奥克兰大学）
Semi-supervised learning has made significant strides in the medical domain since it alleviates the heavy burden of collecting abundant pixel-wise annotated data for semantic segmentation tasks. Existing semi-supervised approaches enhance the ability to extract features from unlabeled data with prior knowledge obtained from limited labeled data. However, due to the scarcity of labeled data, the features extracted by the models are limited in supervised learning, and the quality of predictions for unlabeled data also cannot be guaranteed. Both will impede consistency training. To this end, we p…
Diff-CL: A Novel Cross Pseudo-Supervision Method for Semi-supervised Medical Image Segmentation
School of Mathematical Science（数学科学学院）；Academy for Multidisciplinary Studies（交叉学科研究院）；International Information and Software Institute（国际信息与软件学院）；Dalian University of Technology（大连理工大学）
Efficient Prototype Consistency Learning in Medical Image Segmentation via Joint Uncertainty and Data Augmentation
Faculty of Science and Technology（科技学院）；University of Macau（澳门大学）；Key Laboratory of High Confidence Software Technologies (MOE)（高可信软件技术重点实验室）；School of Computer Science（计算机学院）；Peking University（北京大学）
SemSim: Revisiting Weak-to-Strong Consistency from a Semantic Similarity Perspective for Semi-supervised Medical Image Segmentation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Information Science and Engineering, Ritsumeikan University（立命馆大学信息科学与工程学院）
Yale University（耶鲁大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Automated segmentation in medical image analysis is a challenging task that requires a large amount of manually labeled data. However, most existing learning-based approaches usually suffer from limited manually annotated medical data, which poses a major practical problem for accurate and robust medical image segmentation. In addition, most existing semi-supervised approaches are usually not robust compared with the supervised counterparts, and also lack explicit modeling of geometric structure and semantic information, both of which limit the segmentation accuracy. In this work, we present S…
Clinical Evaluation of Medical Image Synthesis: A Case Study in Wireless Capsule Endoscopy
University of Thessaly（塞萨利大学）；University of Southern Denmark（南丹麦大学）；Pomeranian Medical University（波美拉尼亚医科大学）；Clínica Universidad de Navarra（纳瓦拉大学临床医院）；Mater Dei Hospital（马特尔迪医院）；University of Parma（帕尔马大学）；Hospital Clínic i Provincial de Barcelona（巴塞罗那临床医院）；Centro de Investigación Biomédica en Red de Enfermedades Hepáticas y Digestivas（肝病和消化系统疾病生物医学研究网络中心）；Royal Infirmary of Edinburgh（爱丁堡皇家医院）；University Hospitals of Leicester NHS Trust（莱斯特大学医院NHS信托基金）；Hospital da Senhora da Oliveira - Guimarães（吉马良斯圣母奥利维拉医院）；Lund University（隆德大学）；Skåne University Hospital（斯科讷大学医院）
Mansoura University（曼苏拉大学）；Higher Institute of Engineering and Technology（高等工程技术学院）
Breast cancer is one of the most serious types of cancer that can occur in women. The automatic diagnosis of breast cancer by analyzing histological images (HIs) is important for patients and their prognosis. The classification of HIs provides clinicians with an accurate understanding of diseases and allows them to treat patients more efficiently. Deep learning (DL) approaches have been successfully employed in a variety of fields, particularly medical imaging, due to their capacity to extract features automatically. This study aims to classify different types of breast cancer using HIs. In th…
Argonne National Laboratory（阿贡国家实验室）；Illinois Institute of Technology（伊利诺伊理工学院）；University of Chicago（芝加哥大学）
We explore three representative lines of research and demonstrate the utility of our methods on a classification benchmark of brain cancer MRI data. First, we present a capsule network that explicitly learns a representation robust to rotation and affine transformation. This model requires less training data and outperforms both the original convolutional baseline and a previous capsule network implementation. Second, we leverage the latest domain adaptation techniques to achieve a new state-of-the-art accuracy. Our experiments show that non-medical images can be used to improve model performa…
Indian Statistical Institute（印度统计研究所）；Indira Gandhi National Open University（英迪拉·甘地国立开放大学）；Feinberg School of Medicine, Northwestern University（西北大学范伯格医学院）
Video capsule endoscopy is a hot topic in computer vision and medicine. Deep learning can have a positive impact on the future of video capsule endoscopy technology. It can improve the anomaly detection rate, reduce physicians' time for screening, and aid in real-world clinical analysis. CADx classification system for video capsule endoscopy has shown a great promise for further improvement. For example, detection of cancerous polyp and bleeding can lead to swift medical response and improve the survival rate of the patients. To this end, an automated CADx system must have high throughput and …
The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）
Deep neural networks have achieved remarkable success in a wide variety of natural image and medical image computing tasks. However, these achievements indispensably rely on accurately annotated training data. If encountering some noisy-labeled images, the network training procedure would suffer from difficulties, leading to a sub-optimal classifier. This problem is even more severe in the medical image analysis field, as the annotation quality of medical images heavily relies on the expertise and experience of annotators. In this paper, we propose a novel collaborative training paradigm with …
