Probabilistic Domain Adaptation for Biomedical Image Segmentation
Georg-August-University Göttingen（哥廷根大学）；Institute of Computer Science（计算机科学学院）；Campus Institute Data Science (CIDAS)（校园数据科学研究所）；CAIMed - Lower Saxony Center for AI & Causal Methods in Medicine（下萨克森人工智能与因果医学中心）；Cluster of Excellence Multiscale Bioimaging (MBExC)（多尺度生物成像卓越中心）
Segmentation is a crucial analysis task in biomedical imaging. Given the diverse experimental settings in this field, the lack of generalization limits the use of deep learning in practice. Domain adaptation is a promising remedy: it involves training a model for a given task on a source dataset with labels and adapts it to a target dataset without additional labels. We introduce a probabilistic domain adaptation method, building on self-training approaches and the Probabilistic UNet. We use the latter to sample multiple segmentation hypotheses to implement better pseudo-label filtering. We further study joint and separate source-target training strategies and evaluate our method on three challenging domain adaptation tasks for biomedical segmentation. Our code is publicly available at https://github.com/computational-cell-analytics/Probabilistic-Domain-Adaptation.
The goal of our work is to perform pixel label semantic segmentation on 3D biomedical volumetric data. Manual annotation is always difficult for a large bio-medical dataset. So, we consider two cases where one dataset is fully labeled and the other dataset is assumed to be fully unlabelled. We first perform Semantic Segmentation on the fully labeled isotropic biomedical source data (FIBSEM) and try to incorporate the the trained model for segmenting the target unlabelled dataset(SNEMI3D)which shares some similarities with the source dataset in the context of different types of cellular bodies …
Digitization techniques for biomedical images yield different visual patterns in radiological exams. These differences may hamper the use of data-driven approaches for inference over these images, such as Deep Neural Networks. Another noticeable difficulty in this field is the lack of labeled data, even though in many cases there is an abundance of unlabeled data available. Therefore an important step in improving the generalization capabilities of these methods is to perform Unsupervised and Semi-Supervised Domain Adaptation between different datasets of biomedical images. In order to tackle …
Imperial College London（帝国理工学院）；Microsoft Research Cambridge（微软剑桥研究院）；Cambridge University（剑桥大学）
Significant advances have been made towards building accurate automatic segmentation systems for a variety of biomedical applications using machine learning. However, the performance of these systems often degrades when they are applied on new data that differ from the training data, for example, due to variations in imaging protocols. Manually annotating new data for each test domain is not a feasible solution. In this work we investigate unsupervised domain adaptation using adversarial neural networks to train a segmentation method which is more invariant to differences in the input data, an…
The Chinese University of Hong Kong（香港中文大学）；University of Michigan（密歇根大学）；Imsight Medical Technology Inc.（Imsight医疗科技公司）
Convolutional networks (ConvNets) have achieved great successes in various challenging vision tasks. However, the performance of ConvNets would degrade when encountering the domain shift. The domain adaptation is more significant while challenging in the field of biomedical image analysis, where cross-modality data have largely different distributions. Given that annotating the medical data is especially expensive, the supervised transfer learning approaches are not quite optimal. In this paper, we propose an unsupervised domain adaptation framework with adversarial learning for cross-modality…
School of Medical Imaging, Xuzhou Medical University（徐州医科大学医学影像学院）；The Hong Kong Polytechnic University（香港理工大学）
With the widespread success of deep learning in biomedical image segmentation, domain shift becomes a critical and challenging problem, as the gap between two domains can severely affect model performance when deployed to unseen data with heterogeneous features. To alleviate this problem, we present a novel unsupervised domain adaptation network, for generalizing models learned from the labeled source domain to the unlabeled target domain for cross-modality biomedical image segmentation. Specifically, our approach consists of two key modules, a conditional domain discriminator~(CDD) and a cate…
Domain Adaptation is a technique to address the lack of massive amounts of labeled data in unseen environments. Unsupervised domain adaptation is proposed to adapt a model to new modalities using solely labeled source data and unlabeled target domain data. Though many image-spaces domain adaptation methods have been proposed to capture pixel-level domain-shift, such techniques may fail to maintain high-level semantic information for the segmentation task. For the case of biomedical images, fine details such as blood vessels can be lost during the image transformation operations between domains…
Nanjing University of Posts and Telecommunications（南京邮电大学）
As a more practical setting for unsupervised domain adaptation, Universal Domain Adaptation (UDA) is recently introduced, where the target label set is unknown. One of the big challenges in UDA is how to determine the common label set shared by source and target domains, as there is simply no labeling available in the target domain. In this paper, we employ a probabilistic approach for locating the common label set, where each source class may come from the common label set with a probability. In particular, we propose a novel approach for evaluating the probability of each source class from t…
University of Electronic Science and Technology of China（电子科技大学）；University of Amsterdam（阿姆斯特丹大学）；United Imaging Healthcare, Co., Ltd.（联影医疗科技股份有限公司）
We aim for source-free domain adaptation, where the task is to deploy a model pre-trained on source domains to target domains. The challenges stem from the distribution shift from the source to the target domain, coupled with the unavailability of any source data and labeled target data for optimization. Rather than fine-tuning the model by updating the parameters, we propose to perturb the source model to achieve adaptation to target domains. We introduce perturbations into the model parameters by variational Bayesian inference in a probabilistic framework. By doing so, we can effectively ada…
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；Aalto University（阿尔托大学）；NVIDIA（英伟达）
Source-free domain adaptation (SFDA) aims to adapt a classifier to an unlabelled target data set by only using a pre-trained source model. However, the absence of the source data and the domain shift makes the predictions on the target data unreliable. We propose quantifying the uncertainty in the source model predictions and utilizing it to guide the target adaptation. For this, we construct a probabilistic source model by incorporating priors on the network parameters inducing a distribution over the model predictions. Uncertainties are estimated by employing a Laplace approximation and inco…
Imperial College London（帝国理工学院）；University of Twente（特文特大学）
We present a novel approach for supervised domain adaptation that is based upon the probabilistic framework of Gaussian processes (GPs). Specifically, we introduce domain-specific GPs as local experts for facial expression classification from face images. The adaptation of the classifier is facilitated in probabilistic fashion by conditioning the target expert on multiple source experts. Furthermore, in contrast to existing adaptation approaches, we also learn a target expert from available target data solely. Then, a single and confident classifier is obtained by combining the predictions fro…
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）
In this paper, we aim to solve for unsupervised domain adaptation of classifiers where we have access to label information for the source domain while these are not available for a target domain. While various methods have been proposed for solving these including adversarial discriminator based methods, most approaches have focused on the entire image based domain adaptation. In an image, there would be regions that can be adapted better, for instance, the foreground object may be similar in nature. To obtain such regions, we propose methods that consider the probabilistic certainty estimate …
Microsoft（微软）；IIT Kanpur（印度理工学院坎普尔分校）；Oracle（甲骨文公司）；University of Bath（巴斯大学）
Understanding unsupervised domain adaptation has been an important task that has been well explored. However, the wide variety of methods have not analyzed the role of a classifier's performance in detail. In this paper, we thoroughly examine the role of a classifier in terms of matching source and target distributions. We specifically investigate the classifier ability by matching a) the distribution of features, b) probabilistic uncertainty for samples and c) certainty activation mappings. Our analysis suggests that using these three distributions does result in a consistently improved perfo…
With the FDA approval of Artificial Intelligence (AI) for point-of-care clinical diagnoses, model generalizability is of the utmost importance as clinical decision-making must be domain-agnostic. A method of tackling the problem is to increase the dataset to include images from a multitude of domains; while this technique is ideal, the security requirements of medical data is a major limitation. Additionally, researchers with developed tools benefit from the addition of open-sourced data, but are limited by the difference in domains. Herewith, we investigated the implementation of a Cycle-Cons…
Universidad de Buenos Aires（布宜诺斯艾利斯大学）；Universidad Nacional del Litoral（国立滨海大学）
Automatic segmentation of white matter hyperintensities in magnetic resonance images is of paramount clinical and research importance. Quantification of these lesions serve as a predictor for risk of stroke, dementia and mortality. During the last years, convolutional neural networks (CNN) specifically tailored for biomedical image segmentation have outperformed all previous techniques in this task. However, they are extremely data-dependent, and maintain a good performance only when data distribution between training and test datasets remains unchanged. When such distribution changes but we s…
Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classification
Chongqing University of Technology（重庆理工大学）；Collge of Computer Science, Sichuan University（四川大学计算机学院）；Hebei University of Technology（河北工业大学）；Nanyang Technological University（南洋理工大学）；Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore（新加坡科技研究局前沿人工智能研究中心）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cornell University（康奈尔大学）；Xidian University（西安电子科技大学）
Existing methods for unsupervised domain adaptation often rely on minimizing some statistical distance between the source and target samples in the latent space. To avoid the sampling variability, class imbalance, and data-privacy concerns that often plague these methods, we instead provide a memory and computation-efficient probabilistic framework to extract class prototypes and align the target features with them. We demonstrate the general applicability of our method on a wide range of scenarios, including single-source, multi-source, class-imbalance, and source-private domain adaptation. R…
In the biomedical domain, there is an abundance of dense, complex data where objects of interest may be challenging to detect or constrained by limits of human knowledge. Labelled domain specific datasets for supervised tasks are often expensive to obtain, and furthermore discovery of novel distinct objects may be desirable for unbiased scientific discovery. Therefore, we propose leveraging the wealth of annotations in benchmark computer vision datasets to conduct unsupervised instance segmentation for diverse biomedical datasets. The key obstacle is thus overcoming the large domain shift from…
This paper proposes the CogSense system, which is inspired by sense-making cognition and perception in the mammalian brain to perform perception error detection and perception parameter adaptation using probabilistic signal temporal logic. As a specific application, a contrast-based perception adaption method is presented and validated. The proposed method evaluates perception errors using heterogeneous probe functions computed from the detected objects and subsequently solves a contrast optimization problem to correct perception errors. The CogSense probe functions utilize the characteristics…
We consider the problem of segmenting a biomedical image into anatomical regions of interest. We specifically address the frequent scenario where we have no paired training data that contains images and their manual segmentations. Instead, we employ unpaired segmentation images to build an anatomical prior. Critically these segmentations can be derived from imaging data from a different dataset and imaging modality than the current task. We introduce a generative probabilistic model that employs the learned prior through a convolutional neural network to compute segmentations in an unsupervise…
German Cancer Research Center（德国癌症研究中心）；University of Heidelberg（海德堡大学）；DeepMind；Heidelberg University Hospital（海德堡大学医院）
Biomedical imaging is a driver of scientific discovery and core component of medical care, currently stimulated by the field of deep learning. While semantic segmentation algorithms enable 3D image analysis and quantification in many applications, the design of respective specialised solutions is non-trivial and highly dependent on dataset properties and hardware conditions. We propose nnU-Net, a deep learning framework that condenses the current domain knowledge and autonomously takes the key decisions required to transfer a basic architecture to different datasets and segmentation tasks. Wit…
Basque Center on Cognition, Brain and Language (BCBL)（巴斯克认知、大脑与语言中心（BCBL））；A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院A.A. Martinos生物医学影像中心）
Multi-atlas segmentation (MAS), first introduced and popularized by the pioneering work of Rohlfing, Brandt, Menzel and Maurer Jr (2004), Klein, Mensh, Ghosh, Tourville and Hirsch (2005), and Heckemann, Hajnal, Aljabar, Rueckert and Hammers (2006), is becoming one of the most widely-used and successful image segmentation techniques in biomedical applications. By manipulating and utilizing the entire dataset of "atlases" (training images that have been previously labeled, e.g., manually by an expert), rather than some model-based average representation, MAS has the flexibility to better capture…
Addressing Annotation Scarcity in Hyperspectral Brain Image Segmentation with Unsupervised Domain Adaptation
Technical University of Munich, Munich, Germany（慕尼黑技术大学）；Department of Computing, Imperial College London, London, UK（伦敦帝国学院计算机系）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML））
PVeRA: Probabilistic Vector-Based Random Matrix Adaptation
IHU PRISM, National Center for Precision Medicine in Oncology, Gustave Roussy（IHU PRISM，国家精准医学肿瘤中心，古斯塔夫·罗斯-伊实验室）；Institute of Computer Sciences, CONICET, Universidad de Buenos Aires（计算机科学研究所，CONICET，布宜诺斯艾利斯大学）
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
We developed a new joint probabilistic segmentation and image distribution matching generative adversarial network (PSIGAN) for unsupervised domain adaptation (UDA) and multi-organ segmentation from magnetic resonance (MRI) images. Our UDA approach models the co-dependency between images and their segmentation as a joint probability distribution using a new structure discriminator. The structure discriminator computes structure of interest focused adversarial loss by combining the generated pseudo MRI with probabilistic segmentations produced by a simultaneously trained segmentation sub-networ…
Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation
Institute of Imaging and Computer Vision, RWTH Aachen University（成像与计算机视觉研究所，亚琛RWTH大学）；Bio-manufacturing Engineering Laboratory of Tsinghua SIGS, Tsinghua University（清华大学SIGS生物制造工程实验室）；Shenzhen Eye Hospital（深圳眼科医院）；Department of Ophthalmology, RWTH Aachen University（眼科部，亚琛RWTH大学）
Zhejiang University（浙江大学）；Hikvision Research Institute（海康威视研究院）；National University of Singapore（新加坡国立大学）；Chongqing University（重庆大学）；Hainan Institute of Zhejiang University（浙江大学海南研究院）
Self-training for unsupervised domain adaptive object detection is a challenging task, of which the performance depends heavily on the quality of pseudo boxes. Despite the promising results, prior works have largely overlooked the uncertainty of pseudo boxes during self-training. In this paper, we present a simple yet effective framework, termed as Probabilistic Teacher (PT), which aims to capture the uncertainty of unlabeled target data from a gradually evolving teacher and guides the learning of a student in a mutually beneficial manner. Specifically, we propose to leverage the uncertainty-g…
Finally Outshining the Random Baseline: A Simple and Effective Solution for Active Learning in 3D Biomedical Imaging
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Helmholtz Imaging, German Cancer Research Center (DKFZ)（亥姆霍兹影像，德国癌症研究中心）；University of Heidelberg（海德堡大学）；ETH Zürich（苏黎世联邦理工学院）；Heidelberg University Hospital（海德堡大学医院）；National Center for Tumor Diseases (NCT) Heidelberg（海德堡国家肿瘤疾病中心）
PrAViC: Probabilistic Adaptation Framework for Real-Time Video Classification
Faculty of Mathematics and Computer Science, Jagiellonian University（杰哥利安大学数学与计算机科学学院）；Faculty of Applied Computer Science, University of Information Technology and Management in Rzeszow（里兹诺信息技术与管理大学应用计算机科学学院）；Prometheus MedTech.AI；Doctoral School of Exact and Natural Sciences, Jagiellonian University（杰哥利安大学精确与自然科学博士学院）
We provide a probabilistic interpretation of attention and show that the standard dot-product attention in transformers is a special case of Maximum A Posteriori (MAP) inference. The proposed approach suggests the use of Expectation Maximization algorithms for online adaptation of key and value model parameters. This approach is useful for cases in which external agents, e.g., annotators, provide inference-time information about the correct values of some tokens, e.g, the semantic category of some pixels, and we need for this new information to propagate to other tokens in a principled manner.…
Center for Systems Biology Dresden (CSBD)（德累斯顿系统生物学中心（CSBD））；Max-Planck Institute of Molecular Cell Biology and Genetics（马克斯·普朗克分子细胞生物学与遗传学研究所）
Image denoising is the first step in many biomedical image analysis pipelines and Deep Learning (DL) based methods are currently best performing. A new category of DL methods such as Noise2Void or Noise2Self can be used fully unsupervised, requiring nothing but the noisy data. However, this comes at the price of reduced reconstruction quality. The recently proposed Probabilistic Noise2Void (PN2V) improves results, but requires an additional noise model for which calibration data needs to be acquired. Here, we present improvements to PN2V that (i) replace histogram based noise models by paramet…
