Positive-unlabeled learning for binary and multi-class cell detection in histopathology images with incomplete annotations
Beijing Institute of Technology（北京理工大学）；North China University of Technology（北方工业大学）；Beijing Tiantan Hospital, Capital Medical University（首都医科大学附属北京天坛医院）
Zipei Zhao, Fengqian Pang, Yaou Liu, Zhiwen Liu, Chuyang Ye
备注：
Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2022:027. arXiv admin note: text overlap with arXiv:2106.15918
Cell detection in histopathology images is of great interest to clinical practice and research, and convolutional neural networks (CNNs) have achieved remarkable cell detection results. Typically, to train CNN-based cell detection models, every positive instance in the training images needs to be annotated, and instances that are not labeled as positive are considered negative samples. However, manual cell annotation is complicated due to the large number and diversity of cells, and it can be difficult to ensure the annotation of every positive instance. In many cases, only incomplete annotations are available, where some of the positive instances are annotated and the others are not, and the classification loss term for negative samples in typical network training becomes incorrect. In this work, to address this problem of incomplete annotations, we propose to reformulate the training of the detection network as a positive-unlabeled learning problem. Since the instances in unannotated regions can be either positive or negative, they have unknown labels. Using the samples with unknown labels and the positively labeled samples, we first derive an approximation of the classification loss term corresponding to negative samples for binary cell detection, and based on this approximation we further extend the proposed framework to multi-class cell detection. For evaluation, experiments were performed on four publicly available datasets. The experimental results show that our method improves the performance of cell detection in histopathology images given incomplete annotations for network training.
Beijing Institute of Technology（北京理工大学）；North China University of Technology（北方工业大学）
Cell detection in histopathology images is of great value in clinical practice. \textit{Convolutional neural networks} (CNNs) have been applied to cell detection to improve the detection accuracy, where cell annotations are required for network training. However, due to the variety and large number of cells, complete annotations that include every cell of interest in the training images can be challenging. Usually, incomplete annotations can be achieved, where positive labeling results are carefully examined to ensure their reliability but there can be other positive instances, i.e., cells of …
Semi-supervised Segmentation of Histopathology Images with Noise-Aware Topological Consistency
Stony Brook University（石溪大学）；Athinoula A. Martinos Center for Biomedical Imaging（阿西努拉·A.马蒂诺斯生物医学成像中心）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Histopathology images are gigapixel-sized and include features and information at different resolutions. Collecting annotations in histopathology requires highly specialized pathologists, making it expensive and time-consuming. Self-training can alleviate annotation constraints by learning from both labeled and unlabeled data, reducing the amount of annotations required from pathologists. We study the design of teacher-student self-training systems for Non-alcoholic Steatohepatitis (NASH) using clinical histopathology datasets with limited annotations. We evaluate the models on in-distribution…
To pretrain or not to pretrain? A case study of domain-specific pretraining for semantic segmentation in histopathology
University Of Utah（犹他大学）；Scientific Computing and Imaging Institute（科学计算与成像研究所）；Department of Pathology, University of Utah（犹他大学病理学系）；Kahlert School of Computing, University Of Utah（犹他大学卡尔勒特计算学院）
MATCH: Multi-faceted Adaptive Topo-Consistency for Semi-Supervised Histopathology Segmentation
Stony Brook University（斯通布罗克大学）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）
Histopathology image embedding based on foundation models features aggregation for patient treatment response prediction
Junia（朱尼亚高等学院）；Centrale Lille（中央里尔高等学院）；University of Lille（里尔大学）；Centre for Digital Systems, IMT Nord Europe, Institut Mines-Télécom（北欧电信高等学院数字系统中心、法国矿业电信联盟）；University Hospital of Lille（里尔大学医院）；The University of Tokyo（东京大学）；Yncrea Méditerranée（安克雷亚地中海学院）；Yncrea Ouest（安克雷亚西部学院）
Sunnybrook Research Institute（桑尼布鲁克研究所）；University of Toronto（多伦多大学）
Training a neural network with a large labeled dataset is still a dominant paradigm in computational histopathology. However, obtaining such exhaustive manual annotations is often expensive, laborious, and prone to inter and Intra-observer variability. While recent self-supervised and semi-supervised methods can alleviate this need by learn-ing unsupervised feature representations, they still struggle to generalize well to downstream tasks when the number of labeled instances is small. In this work, we overcome this challenge by leveraging both task-agnostic and task-specific unlabeled data ba…
Rutgers University（罗格斯大学）；SenseBrain Technology Limited LLC（SenseBrain科技有限公司）；Rutgers New Jersey Medical School（罗格斯大学新泽西医学院）；SenseTime Research（商汤科技研究院）；Rutgers Cancer Institute of New Jersey（罗格斯大学新泽西癌症研究所）
Nuclei segmentation is a fundamental task in histopathology image analysis. Typically, such segmentation tasks require significant effort to manually generate accurate pixel-wise annotations for fully supervised training. To alleviate such tedious and manual effort, in this paper we propose a novel weakly supervised segmentation framework based on partial points annotation, i.e., only a small portion of nuclei locations in each image are labeled. The framework consists of two learning stages. In the first stage, we design a semi-supervised strategy to learn a detection model from partially lab…
CoDiR: Confidence-Guided Diffusion Refinement for Semi-Supervised Histopathology Segmentation
AI VIETNAM Lab（AI VIETNAM实验室）；Portland Community College（波特兰社区学院）；University of Science, VNU-HCM（胡志明市国家大学科学大学）；Hanoi University of Science and Technology（河内科技大学）；Jeonbuk National University（全北国立大学）；Washington University School of Medicine（华盛顿大学医学院）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Semi-supervised learning (semi-SL) is a promising alternative to supervised learning for medical image analysis when obtaining good quality supervision for medical imaging is difficult. However, semi-SL assumes that the underlying distribution of unaudited data matches that of the few labeled samples, which is often violated in practical settings, particularly in medical images. The presence of out-of-distribution (OOD) samples in the unlabeled training pool of semi-SL is inevitable and can reduce the efficiency of the algorithm. Common preprocessing methods to filter out outlier samples may n…
University of Notre Dame（圣母大学）；Allen Institute for Cell Science（艾伦细胞科学研究所）
Automatic histopathology image segmentation is crucial to disease analysis. Limited available labeled data hinders the generalizability of trained models under the fully supervised setting. Semi-supervised learning (SSL) based on generative methods has been proven to be effective in utilizing diverse image characteristics. However, it has not been well explored what kinds of generated images would be more useful for model training and how to use such images. In this paper, we propose a new data guided generative method for histopathology image segmentation by leveraging the unlabeled data dist…
This work presents a novel self-supervised pre-training method to learn efficient representations without labels on histopathology medical images utilizing magnification factors. Other state-of-theart works mainly focus on fully supervised learning approaches that rely heavily on human annotations. However, the scarcity of labeled and unlabeled data is a long-standing challenge in histopathology. Currently, representation learning without labels remains unexplored for the histopathology domain. The proposed method, Magnification Prior Contrastive Similarity (MPCS), enables self-supervised lear…
Discrepancy Minimization Improves Cross-Hospital Robustness in Digital Pathology
Reichman University, Herzliya, Israel（以色列海法大学）；Institute of Pathology, Sheba Medical Center, Ramat-Gan, Israel（以色列沙巴医疗中心病理研究所）；Technion - Israel Institute of Technology, Haifa, Israel（以色列技术学院）
A Generative Foundation Model for Multimodal Histopathology
Department of Radiation Oncology, Stanford University School of Medicine（斯坦福大学医学院放射肿瘤学系）；Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine（斯坦福大学医学院精神病学与行为科学系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）；Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Electrical Engineering, Stanford University（斯坦福大学电气工程系）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Department of Radiology, Stanford University（斯坦福大学放射学系）；Center for Artificial Intelligence in Medicine and Imaging, Stanford University（斯坦福大学医学与影像人工智能中心）
GMENet: Generative Mixture of Experts Network for Multi-Center Glioma Diagnosis with Incomplete Imaging Sequences
School of Biomedical Engineering and Technology Innovation, Fudan University（复旦大学生物医学工程与技术创新学院）；Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学脑启发智能科学技术研究院）；Intelligent Diagnosis and Treatment Laboratory for Brain Diseases, Joint Laboratory of Neurosurgery Department of Huashan Hospital and School of Information Science and Technology, Fudan University（脑病智能诊断与治疗实验室，华山医院神经外科部门联合实验室，复旦大学信息科学学院）
MeD-3D: A Multimodal Deep Learning Framework for Precise Recurrence Prediction in Clear Cell Renal Cell Carcinoma (ccRCC)
Skolkovo Institute of Science and Technology (Skoltech)（斯克罗夫诺科学与技术学院（Skoltech））；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
PAII Inc.（PAII公司）；Vanderbilt University（范德堡大学）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）；Ping An Insurance (Group) Company of China, Ltd.（中国平安保险（集团）股份有限公司）；University of Florida（佛罗里达大学）
Acquiring large-scale medical image data, necessary for training machine learning algorithms, is frequently intractable, due to prohibitive expert-driven annotation costs. Recent datasets extracted from hospital archives, e.g., DeepLesion, have begun to address this problem. However, these are often incompletely or noisily labeled, e.g., DeepLesion leaves over 50% of its lesions unlabeled. Thus, effective methods to harvest missing annotations are critical for continued progress in medical image analysis. This is the goal of our work, where we develop a powerful system to harvest missing lesio…
Rethinking Multiple Instance Learning: Developing an Instance-Level Classifier via Weakly-Supervised Self-Training
Digital Medical Research Center, School of Basic Medical Science, Fudan University（复旦大学基础医学院数字医学研究中心）；Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention（上海市医学影像计算与计算机辅助介入重点实验室）
Obtaining a large amount of labeled data in medical imaging is laborious and time-consuming, especially for histopathology. However, it is much easier and cheaper to get unlabeled data from whole-slide images (WSIs). Semi-supervised learning (SSL) is an effective way to utilize unlabeled data and alleviate the need for labeled data. For this reason, we proposed a framework that employs an SSL method to accurately detect cancerous regions with a novel annotation method called Minimal Point-Based annotation, and then utilize the predicted results with an innovative hybrid loss to train a classif…
SenseTime Research（商汤科技研究院）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）；Chinese University of Hong Kong（香港中文大学）
Signet ring cell carcinoma is a type of rare adenocarcinoma with poor prognosis. Early detection leads to huge improvement of patients' survival rate. However, pathologists can only visually detect signet ring cells under the microscope. This procedure is not only laborious but also prone to omission. An automatic and accurate signet ring cell detection solution is thus important but has not been investigated before. In this paper, we take the first step to present a semi-supervised learning framework for the signet ring cell detection problem. Self-training is proposed to deal with the challe…
Tsinghua Shenzhen International Graduate School（清华深圳国际研究生院）；Tsinghua University（清华大学）；Tencent AI Lab（腾讯人工智能实验室）；University of California, Los Angeles（加州大学洛杉矶分校）；The Chinese University of Hong Kong（香港中文大学）
Recently, deep neural networks have greatly advanced histopathology image segmentation but usually require abundant annotated data. However, due to the gigapixel scale of whole slide images and pathologists' heavy daily workload, obtaining pixel-level labels for supervised learning in clinical practice is often infeasible. Alternatively, weakly-supervised segmentation methods have been explored with less laborious image-level labels, but their performance is unsatisfactory due to the lack of dense supervision. Inspired by the recent success of self-supervised learning methods, we present a lab…
CellGenNet: A Knowledge-Distilled Framework for Robust Cell Segmentation in Cancer Tissues
a College of Computing, Georgia Institute of Technology, Atlanta, GA, USA b Department of Pediatrics, Emory University, Atlanta, GA, USA c Departments of Radiology \& Imaging Sciences, Emory University, Atlanta, GA, USA d Department of Biomedical Engineering, Emory University \& Georgia Institute of Technology, Atlanta, GA, USA e School of Electrical
Maximizing T2-Only Prostate Cancer Localization from Expected Diffusion Weighted Imaging
UCL Hawkes Institute and the Department of Medical Physics and Biomedical Engineering, University College London（伦敦大学学院霍克斯研究所与医学物理与生物医学工程系）；Centre for Bioengineering, School of Engineering and Materials Science and Digital Environment Research Institute, Queen Mary University of London（伦敦玛丽女王大学工程与材料科学学院生物工程中心与数字环境研究所）；Centre for Medical Imaging, University College London（伦敦大学学院医学影像中心）；Division of Surgery & Interventional Science, University College London（伦敦大学学院外科与介入科学部）；Urological Research Network（泌尿研究网络）
Large-Scale Pre-training Enables Multimodal AI Differentiation of Radiation Necrosis from Brain Metastasis Progression on Routine MRI
University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根大学医院，埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN)（埃尔朗根-欧洲大都会区纽伦堡综合癌症中心）；Bavarian Cancer Research Center (BZKF)（巴伐利亚癌症研究中心）；Universität Würzburg（维尔茨堡大学）；Peking University（北京大学）；Universitätsklinikum Würzburg（维尔茨堡大学医院）
University Hospital and University of Zurich（苏黎世大学医院和苏黎世大学）
Making histopathology image classifiers robust to a wide range of real-world variability is a challenging task. Here, we describe a candidate deep learning solution for the Mitosis Domain Generalization Challenge 2022 (MIDOG) to address the problem of generalization for mitosis detection in images of hematoxylin-eosin-stained histology slides under high variability (scanner, tissue type and species variability). Our approach consists in training a rotation-invariant deep learning model using aggressive data augmentation with a training set enriched with hard negative examples and automatically…
Stony Brook University（石溪大学）；Snap Inc.（Snap公司）；CentraleSupélec（中央理工-高等电力学院）；University of Paris-Saclay（巴黎萨克雷大学）
Dense prediction tasks such as segmentation and detection of pathological entities hold crucial clinical value in computational pathology workflows. However, obtaining dense annotations on large cohorts is usually tedious and expensive. Contrastive learning (CL) is thus often employed to leverage large volumes of unlabeled data to pre-train the backbone network. To boost CL for dense prediction, some studies have proposed variations of dense matching objectives in pre-training. However, our analysis shows that employing existing dense matching strategies on histopathology images enforces invar…
Self-supervised learning (SSL) enables learning useful inductive biases through utilizing pretext tasks that require no labels. The unlabeled nature of SSL makes it especially important for whole slide histopathological images (WSIs), where patch-level human annotation is difficult. Masked Autoencoders (MAE) is a recent SSL method suitable for digital pathology as it does not require negative sampling and requires little to no data augmentations. However, the domain shift between natural images and digital pathology images requires further research in designing MAE for patch-level WSIs. In thi…
Slide-Level Active Learning Reduces Annotation Burden in H&E images
Institute for Biomedical Informatics, Faculty of Medicine（生物医学信息学研究所，医学学院）；University Hospital Cologne, University of Cologne（科隆大学医院，科隆大学）；Center for Molecular Medicine Cologne (CMMC), Faculty of Medicine（科隆分子医学中心（CMMC），医学学院）；Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne（科隆卓越集群：与衰老相关疾病细胞应激反应（CECAD），科隆大学）；Faculty of Mathematics（数学学院）；Natural Sciences, University of Cologne（自然科学学院，科隆大学）；Institute of Pathology, University Hospital Cologne（病理学研究所，科隆大学医院）
Netherlands Cancer Institute（荷兰癌症研究所）；University of Amsterdam（阿姆斯特丹大学）；VU University Amsterdam（阿姆斯特丹自由大学）；Ellogon AI B.V.（Ellogon AI 公司）；Radboud University Medical Center（拉德堡德大学医学中心）
We present WeakSTIL, an interpretable two-stage weak label deep learning pipeline for scoring the percentage of stromal tumor infiltrating lymphocytes (sTIL%) in H&E-stained whole-slide images (WSIs) of breast cancer tissue. The sTIL% score is a prognostic and predictive biomarker for many solid tumor types. However, due to the high labeling efforts and high intra- and interobserver variability within and between expert annotators, this biomarker is currently not used in routine clinical decision making. WeakSTIL compresses tiles of a WSI using a feature extractor pre-trained with self-supervi…
IBM Research Europe（IBM欧洲研究院）；ETH Zurich（苏黎世联邦理工学院）；Uppsala University（乌普萨拉大学）
Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where…
University of Electronic Science and Technology of China（电子科技大学）；Tokyo Institute of Technology（东京工业大学）
Affective Behavior Analysis is an important part in human-computer interaction. Existing multi-task affective behavior recognition methods suffer from the problem of incomplete labeled datasets. To tackle this problem, this paper presents a semi-supervised model with a mean teacher framework to leverage additional unlabeled data. To be specific, a multi-task model is proposed to learn three different kinds of facial affective representations simultaneously. After that, the proposed model is assigned to be student and teacher networks. When training with unlabeled data, the teacher network is e…
Fudan University（复旦大学）；Hong Kong University of Science and Technology（香港科技大学）；Tencent Youtu Lab（腾讯优图实验室）；Shanghai Jiaotong University（上海交通大学）
Positive-Unlabeled (PU) learning aims to learn a model with rare positive samples and abundant unlabeled samples. Compared with classical binary classification, the task of PU learning is much more challenging due to the existence of many incompletely-annotated data instances. Since only part of the most confident positive samples are available and evidence is not enough to categorize the rest samples, many of these unlabeled data may also be the positive samples. Research on this topic is particularly useful and essential to many real-world tasks which demand very expensive labelling cost. Fo…
Handling Missing Modalities in Multimodal Survival Prediction for Non-Small Cell Lung Cancer
Unit of Artificial Intelligence and Computer Systems, Department of Engineering（人工智能与计算机系统单位，工程系）；Università Campus Bio-Medico di Roma（罗马生物医学大学）；Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering（诊断与介入部门，辐射物理，生物医学工程）；Umeå University（乌梅学院）；Research Unit of Radiation Oncology, Department of Medicine and Surgery（放射肿瘤研究单位，医学与外科部门）；Fondazione Policlinico Universitario Campus Bio-Medico（生物医学大学附属医院基金会）；Anatomical Pathology Operative Research Unit（解剖病理学操作研究单位）；Research Unit of Anatomical Pathology, Department of Medicine and Surgery（解剖病理学研究单位，医学与外科部门）；Department of Medicine and Surgery（医学与外科部门）
In this paper, we study the problem of object counting with incomplete annotations. Based on the observation that in many object counting problems the target objects are normally repeated and highly similar to each other, we are particularly interested in the setting when only a few exemplar annotations are provided. Directly applying object detection with incomplete annotations will result in severe accuracy degradation due to its improper handling of unlabeled object instances. To address the problem, we propose a positiveness-focused object detector (PFOD) to progressively propagate the inc…
Dynamic Entity-Masked Graph Diffusion Model for histopathological image Representation Learning
Xiamen University（厦门大学）；University of Science and Technology of China（中国科学技术大学）；The University of Hong Kong（香港大学）；Univ Montpellier（蒙彼利埃大学）；IMT Mines Ales（法国国立高等矿业电信学校阿莱斯分校）
Leveraging image captions for selective whole slide image annotation
FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）；Technische Hochschule Ingolstadt（英戈尔施塔特应用技术大学）；Flensburg University of Applied Sciences（弗伦斯堡应用技术大学）；Universität Würzburg（维尔茨堡大学）
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）；Georgia Tech（佐治亚理工学院）；Boston University（波士顿大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；UC Berkeley（加州大学伯克利分校）
Training temporal action detection in videos requires large amounts of labeled data, yet such annotation is expensive to collect. Incorporating unlabeled or weakly-labeled data to train action detection model could help reduce annotation cost. In this work, we first introduce the Semi-supervised Action Detection (SSAD) task with a mixture of labeled and unlabeled data and analyze different types of errors in the proposed SSAD baselines which are directly adapted from the semi-supervised classification task. To alleviate the main error of action incompleteness (i.e., missing parts of actions) i…
SemSim: Revisiting Weak-to-Strong Consistency from a Semantic Similarity Perspective for Semi-supervised Medical Image Segmentation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Information Science and Engineering, Ritsumeikan University（立命馆大学信息科学与工程学院）
Virtual histological staining of unlabeled autopsy tissue
University of California, Los Angeles（加州大学洛杉矶分校）；Xi’an Jiaotong University（西安交通大学）；University of California, San Francisco（加州大学旧金山分校）；Hadassah Hebrew University Medical Center（哈达萨希伯来大学医学中心）；University of Southern California（南加州大学）
