Convolutional neural networks (CNNs) have led to significant improvements in the semantic segmentation of images. When source and target datasets come from different modalities, CNN performance suffers due to domain shift. In such cases data annotation in the target domain becomes necessary to maintain model performance. To circumvent the re-annotation process, unsupervised domain adaptation (UDA) is proposed to adapt a model to new modalities using solely unlabeled target data. Common UDA algorithms require access to source domain data during adaptation, which may not be feasible in medical imaging due to data sharing restrictions. In this work, we develop an algorithm for UDA where the source domain data is inaccessible during target adaptation. Our approach is based on encoding the source domain information into an internal distribution that is used to guide adaptation in the absence of source samples. We demonstrate the effectiveness of our algorithm by comparing it to state-of-the-art medical image semantic segmentation approaches on two medical image semantic segmentation datasets.
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Machine learning techniques used in computer-aided medical image analysis usually suffer from the domain shift problem caused by different distributions between source/reference data and target data. As a promising solution, domain adaptation has attracted considerable attention in recent years. The aim of this paper is to survey the recent advances of domain adaptation methods in medical image analysis. We first present the motivation of introducing domain adaptation techniques to tackle domain heterogeneity issues for medical image analysis. Then we provide a review of recent domain adaptati…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Domain adaptation (DA) is an important technique for modern machine learning-based medical data analysis, which aims at reducing distribution differences between different medical datasets. A proper domain adaptation method can significantly enhance the statistical power by pooling data acquired from multiple sites/centers. To this end, we have developed the Domain Adaptation Toolbox for Medical data analysis (DomainATM) - an open-source software package designed for fast facilitation and easy customization of domain adaptation methods for medical data analysis. The DomainATM is implemented in…
Université d'Orléans（奥尔良大学）；Institut National Des Télécommunications et TIC（国家电信与信息通信技术学院）
Predicting the quality of multimedia content is often needed in different fields. In some applications, quality metrics are crucial with a high impact, and can affect decision making such as diagnosis from medical multimedia. In this paper, we focus on such applications by proposing an efficient and shallow model for predicting the quality of medical images without reference from a small amount of annotated data. Our model is based on convolution self-attention that aims to model complex representation from relevant local characteristics of images, which itself slide over the image to interpol…
The success of deep learning has set new benchmarks for many medical image analysis tasks. However, deep models often fail to generalize in the presence of distribution shifts between training (source) data and test (target) data. One method commonly employed to counter distribution shifts is domain adaptation: using samples from the target domain to learn to account for shifted distributions. In this work we propose an unsupervised domain adaptation approach that uses graph neural networks and, disentangled semantic and domain invariant structural features, allowing for better performance acr…
Machine learning algorithms have revolutionized different fields, including natural language processing, computer vision, signal processing, and medical data processing. Despite the excellent capabilities of machine learning algorithms in various tasks and areas, the performance of these models mainly deteriorates when there is a shift in the test and training data distributions. This gap occurs due to the violation of the fundamental assumption that the training and test data are independent and identically distributed (i.i.d). In real-world scenarios where collecting data from all possible d…
Unsupervised domain adaptation (UDA) is one of the key technologies to solve a problem where it is hard to obtain ground truth labels needed for supervised learning. In general, UDA assumes that all samples from source and target domains are available during the training process. However, this is not a realistic assumption under applications where data privacy issues are concerned. To overcome this limitation, UDA without source data, referred to source-free unsupervised domain adaptation (SFUDA) has been recently proposed. Here, we propose a SFUDA method for medical image segmentation. In add…
Domain Adaptation is a technique to address the lack of massive amounts of labeled data in unseen environments. Unsupervised domain adaptation is proposed to adapt a model to new modalities using solely labeled source data and unlabeled target domain data. Though many image-spaces domain adaptation methods have been proposed to capture pixel-level domain-shift, such techniques may fail to maintain high-level semantic information for the segmentation task. For the case of biomedical images, fine details such as blood vessels can be lost during the image transformation operations between domains…
Deep learning-based Computer-Aided Diagnosis (CAD) has attracted appealing attention in academic researches and clinical applications. Nevertheless, the Convolutional Neural Networks (CNNs) diagnosis system heavily relies on the well-labeled lesion dataset, and the sensitivity to the variation of data distribution also restricts the potential application of CNNs in CAD. Unsupervised Domain Adaptation (UDA) methods are developed to solve the expensive annotation and domain gaps problem and have achieved remarkable success in medical image analysis. Yet existing UDA approaches only adapt knowled…
Enhancing and Adapting in the Clinic: Source-free Unsupervised Domain Adaptation for Medical Image Enhancement
Southern University of Science and Technology（南方科技大学）；Agency for Science, Technology and Research（新加坡科技研究局）；Beijing Information Science and Technology University（北京信息科技大学）；University of Science and Technology Beijing（北京科技大学）
South China University of Technology（华南理工大学）；Tencent AI Lab（腾讯人工智能实验室）
Deep learning based medical image diagnosis has shown great potential in clinical medicine. However, it often suffers two major difficulties in practice: 1) only limited labeled samples are available due to expensive annotation costs over medical images; 2) labeled images may contain considerable label noises (e.g., mislabeling labels) due to diagnostic difficulties. In this paper, we seek to exploit rich labeled data from relevant domains to help the learning in the target task with unsupervised domain adaptation (UDA). Unlike most existing UDA methods which rely on clean labeled data or assu…
University of Pennsylvania（宾夕法尼亚大学）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Domain shift, the mismatch between training and testing data characteristics, causes significant degradation in the predictive performance in multi-source imaging scenarios. In medical imaging, the heterogeneity of population, scanners and acquisition protocols at different sites presents a significant domain shift challenge and has limited the widespread clinical adoption of machine learning models. Harmonization methods which aim to learn a representation of data invariant to these differences are the prevalent tools to address domain shift, but they typically result in degradation of predic…
Shanghai Jiao Tong University（上海交通大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；East China Normal University（华东师范大学）
Shape information is a strong and valuable prior in segmenting organs in medical images. However, most current deep learning based segmentation algorithms have not taken shape information into consideration, which can lead to bias towards texture. We aim at modeling shape explicitly and using it to help medical image segmentation. Previous methods proposed Variational Autoencoder (VAE) based models to learn the distribution of shape for a particular organ and used it to automatically evaluate the quality of a segmentation prediction by fitting it into the learned shape distribution. Based on w…
With recent advances in supervised machine learning for medical image analysis applications, the annotated medical image datasets of various domains are being shared extensively. Given that the annotation labelling requires medical expertise, such labels should be applied to as many learning tasks as possible. However, the multi-modal nature of each annotated image renders it difficult to share the annotation label among diverse tasks. In this work, we provide an inductive transfer learning (ITL) approach to adopt the annotation label of the source domain datasets to tasks of the target domain…
AutoMiSeg: Automatic Medical Image Segmentation via Test-Time Adaptation of Foundation Models
Carnegie Mellon University（卡内基梅隆大学）；Brown University（布朗大学）；National Institute of Technology Karnataka（印度卡纳塔克国家理工学院）；University of Pittsburgh（匹兹堡大学）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Curriculum-Based Augmented Fourier Domain Adaptation for Robust Medical Image Segmentation
The Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）；National University of Singapore（新加坡国立大学）；NUSRI Suzhou（新加坡国立大学苏州研究院）；University College London（伦敦大学学院）；Wellcome/EPSRC Centre for Interventional and Surgical Sciences（维康/英国工程与自然科学研究委员会介入与外科科学中心）
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Nankai University（南开大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Southern California（南加州大学）；China University of Mining and Technology（中国矿业大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Recent advances in unsupervised domain adaptation (UDA) show that transferable prototypical learning presents a powerful means for class conditional alignment, which encourages the closeness of cross-domain class centroids. However, the cross-domain inner-class compactness and the underlying fine-grained subtype structure remained largely underexplored. In this work, we propose to adaptively carry out the fine-grained subtype-aware alignment by explicitly enforcing the class-wise separation and subtype-wise compactness with intermediate pseudo labels. Our key insight is that the unlabeled subt…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Unsupervised domain adaptation (UDA) has been a vital protocol for migrating information learned from a labeled source domain to facilitate the implementation in an unlabeled heterogeneous target domain. Although UDA is typically jointly trained on data from both domains, accessing the labeled source domain data is often restricted, due to concerns over patient data privacy or intellectual property. To sidestep this, we propose "off-the-shelf (OS)" UDA (OSUDA), aimed at image segmentation, by adapting an OS segmentor trained in a source domain to a target domain, in the absence of source domai…
Pay Attention to the Atlas: Atlas-Guided Test-Time Adaptation Method for Robust 3D Medical Image Segmentation
School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Imperial College London（伦敦帝国学院）；HeartFlow, Inc.（哈特弗洛公司）；School of Medicine, Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院医学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Computer Science, University of Sheffield（谢菲尔德大学计算机科学系）
University of California Berkeley（加州大学伯克利分校）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
We propose a segmentation framework that uses deep neural networks and introduce two innovations. First, we describe a biophysics-based domain adaptation method. Second, we propose an automatic method to segment white and gray matter, and cerebrospinal fluid, in addition to tumorous tissue. Regarding our first innovation, we use a domain adaptation framework that combines a novel multispecies biophysical tumor growth model with a generative adversarial model to create realistic looking synthetic multimodal MR images with known segmentation. Regarding our second innovation, we propose an automa…
École Centrale de Nantes（南特中央理工学院）；Laboratoire des Sciences du Numérique de Nantes（南特数字科学实验室）；CNRS（法国国家科学研究中心）
This paper addresses the domain shift problem for segmentation. As a solution, we propose OLVA, a novel and lightweight unsupervised domain adaptation method based on a Variational Auto-Encoder (VAE) and Optimal Transport (OT) theory. Thanks to the VAE, our model learns a shared cross-domain latent space that follows a normal distribution, which reduces the domain shift. To guarantee valid segmentations, our shared latent space is designed to model the shape rather than the intensity variations. We further rely on an OT loss to match and align the remaining discrepancy between the two domains …
Due to the lack of properly annotated medical data, exploring the generalization capability of the deep model is becoming a public concern. Zero-shot learning (ZSL) has emerged in recent years to equip the deep model with the ability to recognize unseen classes. However, existing studies mainly focus on natural images, which utilize linguistic models to extract auxiliary information for ZSL. It is impractical to apply the natural image ZSL solutions directly to medical images, since the medical terminology is very domain-specific, and it is not easy to acquire linguistic models for the medical…
Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to an unlabeled target domain. Contrastive learning (CL) in the context of UDA can help to better separate classes in feature space. However, in image segmentation, the large memory footprint due to the computation of the pixel-wise contrastive loss makes it prohibitive to use. Furthermore, labeled target data is not easily available in medical imaging, and obtaining new samples is not economical. As a result, in this work, we tackle a more challenging UDA task when there are only a few (fewsho…
Johns Hopkins University（约翰斯·霍普金斯大学）；Nvidia（英伟达）；University of Florida（佛罗里达大学）；The Chinese University of Hong Kong（香港中文大学）
Although having achieved great success in medical image segmentation, deep learning-based approaches usually require large amounts of well-annotated data, which can be extremely expensive in the field of medical image analysis. Unlabeled data, on the other hand, is much easier to acquire. Semi-supervised learning and unsupervised domain adaptation both take the advantage of unlabeled data, and they are closely related to each other. In this paper, we propose uncertainty-aware multi-view co-training (UMCT), a unified framework that addresses these two tasks for volumetric medical image segmenta…
SDC-UDA: Volumetric Unsupervised Domain Adaptation Framework for Slice-Direction Continuous Cross-Modality Medical Image Segmentation
Yonsei University（延世大学）；Harvard Medical School（哈佛医学院）；Naver AI Lab（Naver人工智能实验室）；Naver Cloud（Naver云）；Probe Medical Inc.（Probe医疗公司）；Martinos Center for Biomedical Imaging（马丁诺斯生物医学成像中心）；Korea Institute of Science and Technology（韩国科学技术研究院）
Domain Adaptation Techniques for Natural and Medical Image Classification
School of Artificial Intelligence, Guilin University of Electronic Technology, Guilin, China, 541004（桂林电子科技大学人工智能学院）；The Laboratory for Imagery, Vision and Artificial Intelligence, Ecole de Technologie Superieure, Montreal, Canada, H3C 1K3（影像、视觉与人工智能实验室，蒙特利尔大学）；College of Computer Science and Engineering, Taibah University, Madinah, Saudi Arabia, 42353（塔布大学计算机科学与工程学院）；College of Computer Science and Engineering, Jeddah University, Jeddah, Saudi Arabia, 23445（吉达大学计算机科学与工程学院）
Digital Medical Research Center, School of Basic Medical Science, Fudan University（复旦大学基础医学院数字医学研究中心）；Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention（上海市医学影像计算与计算机辅助干预重点实验室）
Unsupervised domain adaptation (UDA) aims to learn a model trained on source domain and performs well on unlabeled target domain. In medical image segmentation field, most existing UDA methods depend on adversarial learning to address the domain gap between different image modalities, which is ineffective due to its complicated training process. In this paper, we propose a simple yet effective UDA method based on frequency and spatial domain transfer uner multi-teacher distillation framework. In the frequency domain, we first introduce non-subsampled contourlet transform for identifying domain…
To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire because labeled medical images are not usually available due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions and poor standardization. Lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because …
Fudan University（复旦大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）；Huashan Hospital（华山医院）
Automatic colorectal polyp detection in colonoscopy video is a fundamental task, which has received a lot of attention. Manually annotating polyp region in a large scale video dataset is time-consuming and expensive, which limits the development of deep learning techniques. A compromise is to train the target model by using labeled images and infer on colonoscopy videos. However, there are several issues between the image-based training and video-based inference, including domain differences, lack of positive samples, and temporal smoothness. To address these issues, we propose an Image-video-…
Robust Source-Free Domain Adaptation for Medical Image Segmentation based on Curriculum Learning
Shanghai Digital Medicine Innovation Center, Ruijin Hospital（上海数字医学创新中心，瑞金医院）；Tsinghua University（清华大学）；Medical AI ReSearch (MARS) Group, Guangxi Key Laboratory for Genomic and Personalized Medicine, Guangxi Medical University（医学AI研究组（MARS）, 广西基因组与个性化医学重点实验室，广西医科大学）；Jarvis Research Center, Tencent Youtu Lab（Jarvis研究中心，腾讯优图实验室）；University of California, Berkeley（加州大学伯克利分校）；Westlake University（西湖大学）
The domain discrepancy existed between medical images acquired in different situations renders a major hurdle in deploying pre-trained medical image segmentation models for clinical use. Since it is less possible to distribute training data with the pre-trained model due to the huge data size and privacy concern, source-free unsupervised domain adaptation (SFDA) has recently been increasingly studied based on either pseudo labels or prior knowledge. However, the image features and probability maps used by pseudo label-based SFDA and the consistent prior assumption and the prior prediction netw…
FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation
Beihang University（北京航空航天大学）；Zhejiang Lab（之江实验室）；Singapore University of Technology and Design（新加坡科技设计大学）；BioMind Technology Center（BioMind技术中心）；Zhongguancun Laboratory（中关村实验室）
Aligning What You Separate: Denoised Patch Mixing for Source-Free Domain Adaptation in Medical Image Segmentation
1 Carnegie Mellon University, PA, USA 2 PASSIO Lab, North Carolina A\&T State University, NC, USA 3 Ho Chi Minh University of Science, Vietnam 4 Ho Chi Minh University of Technology, Vietnam 5 Northwestern University, IL, USA
Unified and Semantically Grounded Domain Adaptation for Medical Image Segmentation
Department of Electrical and Computer Engineering, University of Washington（电气与计算机工程系，华盛顿大学）；Department of Bioengineering, University of Washington（生物工程系，华盛顿大学）；Department of Radiology, University of Washington（放射学系，华盛顿大学）；Department of Electrical and Computer Engineering and Paul G. Allen School of Computer Science and Engineering, University of Washington（电气与计算机工程系及保罗·G·阿伦计算机科学与工程学院，华盛顿大学）；Department of Radiology and Imaging Sciences, University of Utah（放射学与影像科学系，犹他大学）
Diffuse-UDA: Addressing Unsupervised Domain Adaptation in Medical Image Segmentation with Appearance and Structure Aligned Diffusion Models
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Guangdong Provincial Key Laboratory of Big Data Computing（广东省大数据计算重点实验室）
QuPID: Quantum Parameter-Efficient Input-Dependent Retrieval Adaptation for Medical RAG
Korea University（高丽大学）；Sookmyung Women’s University（淑明女子大学）；Virginia Tech（弗吉尼亚理工学院暨州立大学）；Seoul National University（首尔大学）；Seoul National University Hospital（首尔大学医院）
DDFP: Data-dependent Frequency Prompt for Source Free Domain Adaptation of Medical Image Segmentation
Digital Medical Research Center, School of Basic Medical Science, Fudan University（复旦大学基础医学学院数字医学研究中心）；Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention（上海医学图像计算与计算机辅助干预重点实验室）
MFM-DA: Instance-Aware Adaptor and Hierarchical Alignment for Efficient Domain Adaptation in Medical Foundation Models
Lanzhou University（兰州大学）；Shanghai Jiaotong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
University of Pittsburgh（匹兹堡大学）；The University of Melbourne（墨尔本大学）；Carnegie Mellon University（卡内基梅隆大学）
Deep learning has achieved remarkable success in medicalimage segmentation, but it usually requires a large numberof images labeled with fine-grained segmentation masks, andthe annotation of these masks can be very expensive andtime-consuming. Therefore, recent methods try to use un-supervised domain adaptation (UDA) methods to borrow in-formation from labeled data from other datasets (source do-mains) to a new dataset (target domain). However, due tothe absence of labels in the target domain, the performance ofUDA methods is much worse than that of the fully supervisedmethod. In this paper, w…
SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation
School of Software, Northwestern Polytechnical University（西北工业大学软件学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；School of Software Engineering, USTC（USTC软件工程学院）；Australian Institute of Health Innovation, Macquarie University（麦考瑞大学健康创新研究所）；Faculty of Applied Science, Macao Polytechnic University（澳门理工学院应用科学学院）；School of Computer Software, Tianjin University（天津大学计算机软件学院）
This work provides a strong baseline for the problem of multi-source multi-target domain adaptation and generalization in medical imaging. Using a diverse collection of ten chest X-ray datasets, we empirically demonstrate the benefits of training medical imaging deep learning models on varied patient populations for generalization to out-of-sample domains.
South China University of Technology（华南理工大学）；Pazhou Lab（琶洲实验室）；Tencent AI Lab（腾讯人工智能实验室）
Deep learning based medical image diagnosis has shown great potential in clinical medicine. However, it often suffers two major difficulties in real-world applications: 1) only limited labels are available for model training, due to expensive annotation costs over medical images; 2) labeled images may contain considerable label noise (e.g., mislabeling labels) due to diagnostic difficulties of diseases. To address these, we seek to exploit rich labeled data from relevant domains to help the learning in the target task via {Unsupervised Domain Adaptation} (UDA). Unlike most UDA methods that rel…
