Automatic histopathology image segmentation is crucial to disease analysis. Limited available labeled data hinders the generalizability of trained models under the fully supervised setting. Semi-supervised learning (SSL) based on generative methods has been proven to be effective in utilizing diverse image characteristics. However, it has not been well explored what kinds of generated images would be more useful for model training and how to use such images. In this paper, we propose a new data guided generative method for histopathology image segmentation by leveraging the unlabeled data distributions. First, we design an image generation module. Image content and style are disentangled and embedded in a clustering-friendly space to utilize their distributions. New images are synthesized by sampling and cross-combining contents and styles. Second, we devise an effective data selection policy for judiciously sampling the generated images: (1) to make the generated training set better cover the dataset, the clusters that are underrepresented in the original training set are covered more; (2) to make the training process more effective, we identify and oversample the images of "hard cases" in the data for which annotated training data may be scarce. Our method is evaluated on glands and nuclei datasets. We show that under both the inductive and transductive settings, our SSL method consistently boosts the performance of common segmentation models and attains state-of-the-art results.
DualProtoSeg: Simple and Efficient Design with Text- and Image-Guided Prototype Learning for Weakly Supervised Histopathology Image Segmentation
University of Houston（德克萨斯大学休斯顿分校）；Ho Chi Minh City University of Technology（胡志明市技术大学）；University of Information Technology（信息技术大学）；University of Technology Sydney（悉尼技术大学）；Vin University（文大学）；Department of Computer Science, Emory University（埃默里大学计算机科学系）；College of Medical Science and Technology, Taipei Medical University（台北医学院医学科学与技术学院）；Montefiore Medical Center, Albert Einstein College of Medicine（蒙特福医院，阿尔伯特·爱因斯坦医学院）；School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院）
Tsinghua Shenzhen International Graduate School（清华深圳国际研究生院）；Tsinghua University（清华大学）；Tencent AI Lab（腾讯人工智能实验室）；University of California, Los Angeles（加州大学洛杉矶分校）；The Chinese University of Hong Kong（香港中文大学）
Recently, deep neural networks have greatly advanced histopathology image segmentation but usually require abundant annotated data. However, due to the gigapixel scale of whole slide images and pathologists' heavy daily workload, obtaining pixel-level labels for supervised learning in clinical practice is often infeasible. Alternatively, weakly-supervised segmentation methods have been explored with less laborious image-level labels, but their performance is unsatisfactory due to the lack of dense supervision. Inspired by the recent success of self-supervised learning methods, we present a lab…
Shenzhen Microbt Electronics Technology Co., Ltd（深圳比特微电子科技有限公司）；Tencent Jarvis Lab（腾讯贾维斯实验室）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Xiamen University（厦门大学）；National Center for Artificial Intelligence, Saudi Data and Artificial Intelligence Authority（沙特数据和人工智能管理局国家人工智能中心）
Semi-supervised learning is a challenging problem which aims to construct a model by learning from limited labeled examples. Numerous methods for this task focus on utilizing the predictions of unlabeled instances consistency alone to regularize networks. However, treating labeled and unlabeled data separately often leads to the discarding of mass prior knowledge learned from the labeled examples. %, and failure to mine the feature interaction between the labeled and unlabeled image pairs. In this paper, we propose a novel method for semi-supervised semantic segmentation named GuidedMix-Net, b…
Southern University of Science and Technology（南方科技大学）；Xiamen University（厦门大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of Artificial Intelligence（因赛普雄人工智能研究院）
Semi-supervised learning is a challenging problem which aims to construct a model by learning from a limited number of labeled examples. Numerous methods have been proposed to tackle this problem, with most focusing on utilizing the predictions of unlabeled instances consistency alone to regularize networks. However, treating labeled and unlabeled data separately often leads to the discarding of mass prior knowledge learned from the labeled examples, and failure to mine the feature interaction between the labeled and unlabeled image pairs. In this paper, we propose a novel method for semi-supe…
GuidedNet: Semi-Supervised Multi-Organ Segmentation via Labeled Data Guide Unlabeled Data
Beihang University（北京航空航天大学）；University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；University of Science and Technology Beijing（北京科技大学）
In this paper, we delve into semi-supervised object detection where unlabeled images are leveraged to break through the upper bound of fully-supervised object detection models. Previous semi-supervised methods based on pseudo labels are severely degenerated by noise and prone to overfit to noisy labels, thus are deficient in learning different unlabeled knowledge well. To address this issue, we propose a data-uncertainty guided multi-phase learning method for semi-supervised object detection. We comprehensively consider divergent types of unlabeled images according to their difficulty levels, …
The Chinese University of Hong Kong（香港中文大学）；SmartMore（思谋科技）
Rapid progress in 3D semantic segmentation is inseparable from the advances of deep network models, which highly rely on large-scale annotated data for training. To address the high cost and challenges of 3D point-level labeling, we present a method for semi-supervised point cloud semantic segmentation to adopt unlabeled point clouds in training to boost the model performance. Inspired by the recent contrastive loss in self-supervised tasks, we propose the guided point contrastive loss to enhance the feature representation and model generalization ability in semi-supervised setting. Semantic p…
Co-Seg++: Mutual Prompt-Guided Collaborative Learning for Versatile Medical Segmentation
School of Computer Science, University of Nottingham Ningbo China（诺丁汉大学宁波校区计算机科学学院）；University of Nottingham, UK（诺丁汉大学（英国））；Graduate School of Information, Production and Systems, Waseda University, Japan（早稻田大学信息、生产与系统研究生院）；Yale University（耶鲁大学）
Inter- and intra-uncertainty based feature aggregation model for semi-supervised histopathology image segmentation
Northwestern Polytechnical University（西北工业大学）；La Trobe University（拉筹伯大学）；CSIRO Data61（联邦科学与工业研究组织数据61部）；University of New South Wales（新南威尔士大学）；Shandong University（山东大学）；Tianjin University（天津大学）
Beijing Institute of Technology（北京理工大学）；North China University of Technology（北方工业大学）
Cell detection in histopathology images is of great value in clinical practice. \textit{Convolutional neural networks} (CNNs) have been applied to cell detection to improve the detection accuracy, where cell annotations are required for network training. However, due to the variety and large number of cells, complete annotations that include every cell of interest in the training images can be challenging. Usually, incomplete annotations can be achieved, where positive labeling results are carefully examined to ensure their reliability but there can be other positive instances, i.e., cells of …
Beijing Institute of Technology（北京理工大学）；North China University of Technology（北方工业大学）；Beijing Tiantan Hospital, Capital Medical University（首都医科大学附属北京天坛医院）
Cell detection in histopathology images is of great interest to clinical practice and research, and convolutional neural networks (CNNs) have achieved remarkable cell detection results. Typically, to train CNN-based cell detection models, every positive instance in the training images needs to be annotated, and instances that are not labeled as positive are considered negative samples. However, manual cell annotation is complicated due to the large number and diversity of cells, and it can be difficult to ensure the annotation of every positive instance. In many cases, only incomplete annotati…
Most progress in semantic segmentation reports on daytime images taken under favorable illumination conditions. We instead address the problem of semantic segmentation of nighttime images and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night via labeled synthetic images and unlabeled real images, bo…
Beihang University（北京航空航天大学）；Hefei Innovation Research Institute, Beihang University（北京航空航天大学合肥创新研究院）；Beijing Advanced Innovation Centre for Biomedical Engineering（北京生物医学工程高精尖创新中心）
Medical image segmentation is a fundamental and critical step in many clinical approaches. Semi-supervised learning has been widely applied to medical image segmentation tasks since it alleviates the heavy burden of acquiring expert-examined annotations and takes the advantage of unlabeled data which is much easier to acquire. Although consistency learning has been proven to be an effective approach by enforcing an invariance of predictions under different distributions, existing approaches cannot make full use of region-level shape constraint and boundary-level distance information from unlab…
Grading of cancer is important to know the extent of its spread. Prior to grading, segmentation of glandular structures is important. Manual segmentation is a time consuming process and is subject to observer bias. Hence, an automated process is required to segment the gland structures. These glands show a large variation in shape size and texture. This makes the task challenging as the glands cannot be segmented using mere morphological operations and conventional segmentation mechanisms. In this project we propose a method which detects the boundary epithelial cells of glands and then a nove…
ThreshGuide: Class-Aware Labeled-Guided Thresholding for Semi-Supervised 3D Abdominal Multi-Organ Segmentation
School of Intelligent Science and Technology, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院智能科学与技术学院）
Barcelona Supercomputing Center（巴塞罗那超级计算中心）；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）
Image segmentation methods are usually trained with pixel-level annotations, which require significant human effort to collect. The most common solution to address this constraint is to implement weakly-supervised pipelines trained with lower forms of supervision, such as bounding boxes or scribbles. Another option are semi-supervised methods, which leverage a large amount of unlabeled data and a limited number of strongly-labeled samples. In this second setup, samples to be strongly-annotated can be selected randomly or with an active learning mechanism that chooses the ones that will maximiz…
We address the problem of semantic nighttime image segmentation and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night through progressively darker times of day, exploiting cross-time-of-day correspondences between daytime images from a reference map and dark images to guide the label inference in th…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences（中国科学院脑科学与智能技术卓越创新中心）
3D point cloud semantic segmentation is a challenging topic in the computer vision field. Most of the existing methods in literature require a large amount of fully labeled training data, but it is extremely time-consuming to obtain these training data by manually labeling massive point clouds. Addressing this problem, we propose a superpoint-guided semi-supervised segmentation network for 3D point clouds, which jointly utilizes a small portion of labeled scene point clouds and a large number of unlabeled point clouds for network training. The proposed network is iteratively updated with its p…
Existing deep neural networks for histopathology image synthesis cannot generate image styles that align with different organs, and cannot produce accurate boundaries of clustered nuclei. To address these issues, we propose a style-guided instance-adaptive normalization (SIAN) approach to synthesize realistic color distributions and textures for histopathology images from different organs. SIAN contains four phases, semantization, stylization, instantiation, and modulation. The first two phases synthesize image semantics and styles by using semantic maps and learned image style vectors. The in…
Semi-supervised Segmentation of Histopathology Images with Noise-Aware Topological Consistency
Stony Brook University（石溪大学）；Athinoula A. Martinos Center for Biomedical Imaging（阿西努拉·A.马蒂诺斯生物医学成像中心）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（厦门大学多媒体可信感知与高效计算教育部重点实验室）；Contemporary Amperex Technology Co. Limited（宁德时代新能源科技股份有限公司）
Histopathology images are gigapixel-sized and include features and information at different resolutions. Collecting annotations in histopathology requires highly specialized pathologists, making it expensive and time-consuming. Self-training can alleviate annotation constraints by learning from both labeled and unlabeled data, reducing the amount of annotations required from pathologists. We study the design of teacher-student self-training systems for Non-alcoholic Steatohepatitis (NASH) using clinical histopathology datasets with limited annotations. We evaluate the models on in-distribution…
University Hospital Essen (AöR)（埃森大学医院）；Cancer Research Center Cologne Essen (CCCE)（科隆-埃森癌症研究中心）；West German Cancer Center Essen（埃森西德癌症中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；University of Duisburg-Essen（杜伊斯堡-埃森大学）；Hannover Medical School（汉诺威医学院）
The segmentation of histopathological whole slide images into tumourous and non-tumourous types of tissue is a challenging task that requires the consideration of both local and global spatial contexts to classify tumourous regions precisely. The identification of subtypes of tumour tissue complicates the issue as the sharpness of separation decreases and the pathologist's reasoning is even more guided by spatial context. However, the identification of detailed types of tissue is crucial for providing personalized cancer therapies. Due to the high resolution of whole slide images, existing sem…
UCAD: Uncertainty-guided Contour-aware Displacement for semi-supervised medical image segmentation
University of Science and Technology of China（科学技术大学）；Suzhou Institute for Advanced Research（苏州先进研究院）；State Key Laboratory of Precision and Intelligent Chemistry（精密与智能化学国家重点实验室）
College of Computer Science and Technology, Huaqiao University（华侨大学计算机科学与技术学院）
Semantic segmentation is an essential step for electron microscopy (EM) image analysis. Although supervised models have achieved significant progress, the need for labor intensive pixel-wise annotation is a major limitation. To complicate matters further, supervised learning models may not generalize well on a novel dataset due to domain shift. In this study, we introduce an adversarial-prediction guided multi-task network to learn the adaptation of a well-trained model for use on a novel unlabeled target domain. Since no label is available on target domain, we learn an encoding representation…
CoDiR: Confidence-Guided Diffusion Refinement for Semi-Supervised Histopathology Segmentation
AI VIETNAM Lab（AI VIETNAM实验室）；Portland Community College（波特兰社区学院）；University of Science, VNU-HCM（胡志明市国家大学科学大学）；Hanoi University of Science and Technology（河内科技大学）；Jeonbuk National University（全北国立大学）；Washington University School of Medicine（华盛顿大学医学院）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation
AI VIETNAM Lab（AI越南实验室）；Washington University School of Medicine（华盛顿大学医学院）；Jeonbuk National University（全北国立大学）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Chinese Academy of Military Science（中国军事科学院）；National University of Defense Technology（国防科技大学）
As a powerful way of realizing semi-supervised segmentation, the cross supervision method learns cross consistency based on independent ensemble models using abundant unlabeled images. However, the wrong pseudo labeling information generated by cross supervision would confuse the training process and negatively affect the effectiveness of the segmentation model. Besides, the training process of ensemble models in such methods also multiplies the cost of computation resources and decreases the training efficiency. To solve these problems, we propose a novel cross supervision method, namely unce…
Peking University（北京大学）；Deepwise AI Lab（深睿医疗人工智能实验室）；Microsoft Research Asia（微软亚洲研究院）；The University of Hong Kong（香港大学）
As an important upstream task for many medical applications, supervised landmark localization still requires non-negligible annotation costs to achieve desirable performance. Besides, due to cumbersome collection procedures, the limited size of medical landmark datasets impacts the effectiveness of large-scale self-supervised pre-training methods. To address these challenges, we propose a two-stage framework for one-shot medical landmark localization, which first infers landmarks by unsupervised registration from the labeled exemplar to unlabeled targets, and then utilizes these noisy pseudo l…
Pseudo Label-Guided Data Fusion and Output Consistency for Semi-Supervised Medical Image Segmentation
college of physics and information engineering, University of Fuzhou University（福州大学物理与信息工程学院）；Faculty of Applied Science, University of Macao Polytechnic University（澳门理工大学应用科学学院）；University of Toronto University（多伦多大学）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Medicine Essen（埃森大学医学院）
Producing densely annotated data is a difficult and tedious task for medical imaging applications. To address this problem, we propose a novel approach to generate supervision for semi-supervised semantic segmentation. We argue that visually similar regions between labeled and unlabeled images likely contain the same semantics and therefore should share their label. Following this thought, we use a small number of labeled images as reference material and match pixels in an unlabeled image to the semantics of the best fitting pixel in a reference set. This way, we avoid pitfalls such as confirm…
Faculty of Informatics and Data Science, University of Regensburg, Germany（信息学与数据科学学院，雷根萨大学）；Indian Institute of Information Technology Bhopal, India（比哈尔信息学院）
Most modern approaches for domain adaptive semantic segmentation rely on continued access to source data during adaptation, which may be infeasible due to computational or privacy constraints. We focus on source-free domain adaptation for semantic segmentation, wherein a source model must adapt itself to a new target domain given only unlabeled target data. We propose Augmentation Consistency-guided Self-training (AUGCO), a source-free adaptation algorithm that uses the model's pixel-level predictive consistency across diverse, automatically generated views of each target image along with mode…
AWGUNET: Attention-Aided Wavelet Guided U-Net for Nuclei Segmentation in Histopathology Images
Jadavpur University（jadavpur大学）；Artificial Intelligence Research Institute, China University of Mining and Technology（中国矿业大学人工智能研究院）；Saint Petersburg Electrotechnical University "LETI"（圣彼得堡国立电工技术大学"LETI"）
Semi-supervised learning (semi-SL) is a promising alternative to supervised learning for medical image analysis when obtaining good quality supervision for medical imaging is difficult. However, semi-SL assumes that the underlying distribution of unaudited data matches that of the few labeled samples, which is often violated in practical settings, particularly in medical images. The presence of out-of-distribution (OOD) samples in the unlabeled training pool of semi-SL is inevitable and can reduce the efficiency of the algorithm. Common preprocessing methods to filter out outlier samples may n…
To pretrain or not to pretrain? A case study of domain-specific pretraining for semantic segmentation in histopathology
University Of Utah（犹他大学）；Scientific Computing and Imaging Institute（科学计算与成像研究所）；Department of Pathology, University of Utah（犹他大学病理学系）；Kahlert School of Computing, University Of Utah（犹他大学卡尔勒特计算学院）
Clinically-Informed Modeling for Pediatric Brain Tumor Classification from Whole-Slide Histopathology Images
Dept. of Computer Science（计算机科学系）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；School of Information（信息学院）；Dell Children's Medical Center（德尔儿童医学中心）；Dept. of OREI（OREI部门）；University of Nevada, Reno（内华达大学里诺分校）
MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features
Heilongjiang University（黑龙江大学）；La Trobe University（拉筹伯大学）；University of South China（南华大学）；The First Affiliated Hospital（第一附属医院）；Hengyang Medical School（衡阳医学院）；School of Basic Medical Sciences（基础医学院）
Retrieval-Guided Generation for Safer Histopathology Image Captioning
Kimia Lab, Department of Artificial Intelligence \& Informatics, Mayo Clinic, Rochester, MN, USA；Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, USA；Lane Department of Computer Science；Electrical Engineering, West Virginia University, Morgantown, WV, USA；Department of Computer Science；Engineering, Princeton University, Princeton, NJ, USA；Department of Computer Sciences, University of Wisconsin--Madison, Madison, WI, USA
Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation
Fuzhou University（福州大学）；Fujian Medical University Union Hospital（福建医科大学附属协和医院）；Fujian Medical Ultrasound Research Institute（福建省医学超声研究所）；Macao Polytechnic University（澳门理工大学）
MATCH: Multi-faceted Adaptive Topo-Consistency for Semi-Supervised Histopathology Segmentation
Stony Brook University（斯通布罗克大学）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）
MCICSAM: Monte Carlo-guided Interpolation Consistency Segment Anything Model for Semi-Supervised Prostate Zone Segmentation
College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学与生物信息工程学院）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；University of Chicago（芝加哥大学）；University of Dundee（邓迪大学）；School of Health Management, China Medical University（中国医科大学健康管理学院）
