Test time Adaptation is a promising approach for mitigating domain shift in medical image segmentation; however, current evaluations remain limited in terms of modality coverage, task diversity, and methodological consistency. We present MedSeg-TTA, a comprehensive benchmark that examines twenty representative adaptation methods across seven imaging modalities, including MRI, CT, ultrasound, pathology, dermoscopy, OCT, and chest X-ray, under fully unified data preprocessing, backbone configuration, and test time protocols. The benchmark encompasses four significant adaptation paradigms: Input-level Transformation, Feature-level Alignment, Output-level Regularization, and Prior Estimation, enabling the first systematic cross-modality comparison of their reliability and applicability. The results show that no single paradigm performs best in all conditions. Input-level methods are more stable under mild appearance shifts. Feature-level and Output-level methods offer greater advantages in boundary-related metrics, whereas prior-based methods exhibit strong modality dependence. Several methods degrade significantly under large inter-center and inter-device shifts, which highlights the importance of principled method selection for clinical deployment. MedSeg-TTA provides standardized datasets, validated implementations, and a public leaderboard, establishing a rigorous foundation for future research on robust, clinically reliable test-time adaptation. All source codes and open-source datasets are available at https://github.com/wenjing-gg/MedSeg-TTA.
Convolutional neural networks have been applied to a wide variety of computer vision tasks. Recent advances in semantic segmentation have enabled their application to medical image segmentation. While most CNNs use two-dimensional kernels, recent CNN-based publications on medical image segmentation featured three-dimensional kernels, allowing full access to the three-dimensional structure of medical images. Though closely related to semantic segmentation, medical image segmentation includes specific challenges that need to be addressed, such as the scarcity of labelled data, the high class imb…
Tianjin University（天津大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）
Segmentation is a fundamental task in medical image analysis. However, most existing methods focus on primary region extraction and ignore edge information, which is useful for obtaining accurate segmentation. In this paper, we propose a generic medical segmentation method, called Edge-aTtention guidance Network (ET-Net), which embeds edge-attention representations to guide the segmentation network. Specifically, an edge guidance module is utilized to learn the edge-attention representations in the early encoding layers, which are then transferred to the multi-scale decoding layers, fused usin…
School of Mechatronic Engineering and Automation, Shanghai University（上海大学机电工程与自动化学院）；Cixi Institute of Biomedical Engineering, Chinese Academy of Sciences（中国科学院慈溪生物医学工程研究所）；Inception Institute of Artificial Intelligence（Inception人工智能研究所）；Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore（新加坡科技研究局信息通信研究院）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）
Medical image segmentation is an important step in medical image analysis. With the rapid development of convolutional neural network in image processing, deep learning has been used for medical image segmentation, such as optic disc segmentation, blood vessel detection, lung segmentation, cell segmentation, etc. Previously, U-net based approaches have been proposed. However, the consecutive pooling and strided convolutional operations lead to the loss of some spatial information. In this paper, we propose a context encoder network (referred to as CE-Net) to capture more high-level information…
The Chinese University of Hong Kong（香港中文大学）；Stanford University（斯坦福大学）
Deep convolutional neural networks have achieved remarkable progress on a variety of medical image computing tasks. A common problem when applying supervised deep learning methods to medical images is the lack of labeled data, which is very expensive and time-consuming to be collected. In this paper, we present a novel semi-supervised method for medical image segmentation, where the network is optimized by the weighted combination of a common supervised loss for labeled inputs only and a regularization loss for both labeled and unlabeled data. To utilize the unlabeled data, our method encourag…
SimTxtSeg: Weakly-Supervised Medical Image Segmentation with Simple Text Cues
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, China（教育部新一代人工智能技术及交叉应用重点实验室）；Nanjing University of Science and Technology（南京理工大学）；Northwestern Polytechnical University（西北工业大学）
Kyushu University（九州大学）；Kyoto University（京都大学）；The University of Tokyo（东京大学）
Semi-supervised domain adaptation is a technique to build a classifier for a target domain by modifying a classifier in another (source) domain using many unlabeled samples and a small number of labeled samples from the target domain. In this paper, we develop a semi-supervised domain adaptation method, which has robustness to class-imbalanced situations, which are common in medical image classification tasks. For robustness, we propose a weakly-supervised clustering pipeline to obtain high-purity clusters and utilize the clusters in representation learning for domain adaptation. The proposed …
UiT The Arctic University of Norway（特罗姆瑟大学（挪威北极圈大学））
Recent work has shown that label-efficient few-shot learning through self-supervision can achieve promising medical image segmentation results. However, few-shot segmentation models typically rely on prototype representations of the semantic classes, resulting in a loss of local information that can degrade performance. This is particularly problematic for the typically large and highly heterogeneous background class in medical image segmentation problems. Previous works have attempted to address this issue by learning additional prototypes for each class, but since the prototypes are based on…
University of Electronic Science and Technology of China（电子科技大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Johns Hopkins University（约翰斯·霍普金斯大学）；West China Hospital, Sichuan University（四川大学华西医院）
Background and Objective: Open-source deep learning toolkits are one of the driving forces for developing medical image segmentation models. Existing toolkits mainly focus on fully supervised segmentation and require full and accurate pixel-level annotations that are time-consuming and difficult to acquire for segmentation tasks, which makes learning from imperfect labels highly desired for reducing the annotation cost. We aim to develop a new deep learning toolkit to support annotation-efficient learning for medical image segmentation. Methods: Our proposed toolkit named PyMIC is a modular de…
LIPN-UMR-CNRS 7030（法国国家科学研究中心7030联合研究单元LIPN实验室）；Sorbonne University Paris Nord（巴黎北索邦大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）；Laboratory of Images, Signals and Intelligent Systems（图像、信号与智能系统实验室）；Université Paris-Est Créteil（巴黎东克雷泰伊大学）
Manual segmentation of medical images (e.g., segmenting tumors in CT scans) is a high-effort task that can be accelerated with machine learning techniques. However, selecting the right segmentation approach depends on the evaluation function, particularly in medical image segmentation where we must deal with dependency between voxels. For instance, in contrast to classical systems where the predictions are either correct or incorrect, predictions in medical image segmentation may be partially correct and incorrect simultaneously. In this paper, we explore this expressiveness to extract the use…
Beihang University（北京航空航天大学）；Hefei Innovation Research Institute, Beihang University（北京航空航天大学合肥创新研究院）；Beijing Advanced Innovation Centre for Biomedical Engineering（北京生物医学工程先进创新中心）；Beijing Advanced Innovation Centre for Big Data-Based Precision Medicine（北京大数据精准医疗先进创新中心）
Recently, deep convolutional neural networks have achieved great success for medical image segmentation. However, unlike segmentation of natural images, most medical images such as MRI and CT are volumetric data. In order to make full use of volumetric information, 3D CNNs are widely used. However, 3D CNNs suffer from higher inference time and computation cost, which hinders their further clinical applications. Additionally, with the increased number of parameters, the risk of overfitting is higher, especially for medical images where data and annotations are expensive to acquire. To issue thi…
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）
Deep convolutional neural networks (CNNs) have been widely used for medical image segmentation. In most studies, only the output layer is exploited to compute the final segmentation results and the hidden representations of the deep learned features have not been well understood. In this paper, we propose a prototype segmentation (ProtoSeg) method to compute a binary segmentation map based on deep features. We measure the segmentation abilities of the features by computing the Dice between the feature segmentation map and ground-truth, named as the segmentation ability score (SA score for shor…
Institute for Machine Learning, ETH Zurich（苏黎世联邦理工学院机器学习研究所）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院）；Champalimaud Foundation（尚帕利莫基金会）
In clinical practice, regions of interest in medical imaging often need to be identified through a process of precise image segmentation. The quality of this image segmentation step critically affects the subsequent clinical assessment of the patient status. To enable high accuracy, automatic image segmentation, we introduce a novel deep neural network architecture that exploits the inherent spatial coherence of anatomical structures and is well equipped to capture long-range spatial dependencies in the segmented pixel/voxel space. In contrast to the state-of-the-art solutions based on convolu…
This work provides a strong baseline for the problem of multi-source multi-target domain adaptation and generalization in medical imaging. Using a diverse collection of ten chest X-ray datasets, we empirically demonstrate the benefits of training medical imaging deep learning models on varied patient populations for generalization to out-of-sample domains.
University of Science and Technology Beijing（北京科技大学）；University of Central Florida（中佛罗里达大学）
Transformer, benefiting from global (long-range) information modeling using self-attention mechanism, has been successful in natural language processing and computer vision recently. Convolutional Neural Networks, capable of capturing local features, are difficult to model explicit long-distance dependencies from global feature space. However, both local and global features are crucial for dense prediction tasks, especially for 3D medical image segmentation. In this paper, we present the further attempt to exploit Transformer in 3D CNN for 3D medical image volumetric segmentation and propose a…
Vanderbilt University（范德堡大学）；The George Washington University（乔治华盛顿大学）；Korea University（高丽大学）
Segmenting medical images accurately and reliably is important for disease diagnosis and treatment. It is a challenging task because of the wide variety of objects' sizes, shapes, and scanning modalities. Recently, many convolutional neural networks (CNN) have been designed for segmentation tasks and achieved great success. Few studies, however, have fully considered the sizes of objects, and thus most demonstrate poor performance for small objects segmentation. This can have a significant impact on the early detection of diseases. This paper proposes a Context Axial Reserve Attention Network …
South China University of Technology（华南理工大学）；Pazhou Lab（琶洲实验室）；Tencent AI Lab（腾讯人工智能实验室）
Deep learning based medical image diagnosis has shown great potential in clinical medicine. However, it often suffers two major difficulties in real-world applications: 1) only limited labels are available for model training, due to expensive annotation costs over medical images; 2) labeled images may contain considerable label noise (e.g., mislabeling labels) due to diagnostic difficulties of diseases. To address these, we seek to exploit rich labeled data from relevant domains to help the learning in the target task via {Unsupervised Domain Adaptation} (UDA). Unlike most UDA methods that rel…
South China University of Technology（华南理工大学）；Rochester Institute of Technology（罗切斯特理工学院）
We introduce a novel frame-interpolation-based method for slice imputation to improve segmentation accuracy for anisotropic 3D medical images, in which the number of slices and their corresponding segmentation labels can be increased between two consecutive slices in anisotropic 3D medical volumes. Unlike previous inter-slice imputation methods, which only focus on the smoothness in the axial direction, this study aims to improve the smoothness of the interpolated 3D medical volumes in all three directions: axial, sagittal, and coronal. The proposed multitask inter-slice imputation method, in …
Vanderbilt University（范德比尔特大学）；The George Washington University（乔治华盛顿大学）
Segmenting medical images accurately and reliably is important for disease diagnosis and treatment. It is a challenging task because of the wide variety of objects' sizes, shapes, and scanning modalities. Recently, many convolutional neural networks (CNN) have been designed for segmentation tasks and achieved great success. Few studies, however, have fully considered the sizes of objects, and thus most demonstrate poor performance for small objects segmentation. This can have a significant impact on the early detection of diseases. This paper proposes a Context Axial Reverse Attention Network …
From CNN to Transformer: A Review of Medical Image Segmentation Models
University of Electronic Science and Technology of China（电子科技大学）；Deyang People’s Hospital（德阳市人民医院）；Sichuan Provincial People’s Hospital（四川省人民医院）；Chinese Academy of Sciences Sichuan Translational Medicine Research Hospital（中国科学院四川转化医学研究医院）
UNSW（新南威尔士大学）；Dongguk University（东国大学）；Defence Science and Technology Group（国防科学技术集团）
Image segmentation is an important task in medical imaging. It constitutes the backbone of a wide variety of clinical diagnostic methods, treatments, and computer-aided surgeries. In this paper, we propose a multi-kernel image segmentation net (MKIS-Net), which uses multiple kernels to create an efficient receptive field and enhance segmentation performance. As a result of its multi-kernel design, MKIS-Net is a light-weight architecture with a small number of trainable parameters. Moreover, these multi-kernel receptive fields also contribute to better segmentation results. We demonstrate the e…
University of Notre Dame（圣母大学）；The Hong Kong Polytechnic University（香港理工大学）
Segmentation maps of medical images annotated by medical experts contain rich spatial information. In this paper, we propose to decompose annotation maps to learn disentangled and richer feature transforms for segmentation problems in medical images. Our new scheme consists of two main stages: decompose and integrate. Decompose: by annotation map decomposition, the original segmentation problem is decomposed into multiple segmentation sub-problems; these new segmentation sub-problems are modeled by training multiple deep learning modules, each with its own set of feature transforms. Integrate:…
SQA-SAM: Segmentation Quality Assessment for Medical Images Utilizing the Segment Anything Model
Nanjing University of Science and Technology（南京理工大学）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医学研究中心）；The Chinese University of Hong Kong（香港中文大学）；University of Notre Dame（圣母大学）
The scarcity of pixel-level annotation is a prevalent problem in medical image segmentation tasks. In this paper, we introduce a novel regularization strategy involving interpolation-based mixing for semi-supervised medical image segmentation. The proposed method is a new consistency regularization strategy that encourages segmentation of interpolation of two unlabelled data to be consistent with the interpolation of segmentation maps of those data. This method represents a specific type of data-adaptive regularization paradigm which aids to minimize the overfitting of labelled data under high…
Flordia International University（佛罗里达国际大学）；University of Notre Dame（圣母大学）；Northeastern University（东北大学）；Guangdong General Hospital（广东省人民医院）
Cloud based medical image analysis has become popular recently due to the high computation complexities of various deep neural network (DNN) based frameworks and the increasingly large volume of medical images that need to be processed. It has been demonstrated that for medical images the transmission from local to clouds is much more expensive than the computation in the clouds itself. Towards this, 3D image compression techniques have been widely applied to reduce the data traffic. However, most of the existing image compression techniques are developed around human vision, i.e., they are de…
Macquarie University（麦考瑞大学）；University of Technology Sydney（悉尼科技大学）；The University of Sydney（悉尼大学）
Automatic tumor segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on convolutional neural networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because regular CNNs can only exploit translation invariance, ignoring further inherent symmetries existing in medical images such as rotations and reflections. To mitigate this shortcoming, we propose a novel group equivariant segmentation framework by encoding those inherent symmetries for learning mor…
In-context learning for medical image segmentation
Tohoku University Graduate School of Medicine（东北大学大学院医学系研究科）；Tohoku University Hospital（东北大学医院）；St. Marianna University Graduate School of Medicine（圣玛丽安娜医科大学大学院医学系研究科）
University of Georgia（佐治亚大学）；Zhongnan Hospital of Wuhan University（武汉大学中南医院）；Shanghai Jiao Tong University（上海交通大学）
In this paper we consider the problem of unsupervised anomaly segmentation in medical images, which has attracted increasing attention in recent years due to the expensive pixel-level annotations from experts and the existence of a large amount of unannotated normal and abnormal image scans. We introduce a segmentation network that utilizes adversarial learning to partition an image into two cuts, with one of them falling into a reference distribution provided by the user. This Adversarial-based Selective Cutting network (ASC-Net) bridges the two domains of cluster-based deep segmentation and …
University of Cambridge（剑桥大学）；University of Salerno（萨莱诺大学）
Automatic segmentation methods are an important advancement in medical image analysis. Machine learning techniques, and deep neural networks in particular, are the state-of-the-art for most medical image segmentation tasks. Issues with class imbalance pose a significant challenge in medical datasets, with lesions often occupying a considerably smaller volume relative to the background. Loss functions used in the training of deep learning algorithms differ in their robustness to class imbalance, with direct consequences for model convergence. The most commonly used loss functions for segmentati…
University of Augsburg（奥格斯堡大学）；University Hospital Augsburg（奥格斯堡大学医院）
Neural networks represent a field of research that can quickly achieve very good results in the field of medical image segmentation using a GPU. A possible way to achieve good results without GPUs are random forests. For this purpose, two random forest approaches were compared with a state-of-the-art deep convolutional neural network. To make the comparison the PhC-C2DH-U373 and the retinal imaging datasets were used. The evaluation showed that the deep convolutional neutral network achieved the best results. However, one of the random forest approaches also achieved a similar high performance…
In this paper, we present UNet++, a new, more powerful architecture for medical image segmentation. Our architecture is essentially a deeply-supervised encoder-decoder network where the encoder and decoder sub-networks are connected through a series of nested, dense skip pathways. The re-designed skip pathways aim at reducing the semantic gap between the feature maps of the encoder and decoder sub-networks. We argue that the optimizer would deal with an easier learning task when the feature maps from the decoder and encoder networks are semantically similar. We have evaluated UNet++ in compari…
Recent advances of semantic image segmentation greatly benefit from deeper and larger Convolutional Neural Network (CNN) models. Compared to image segmentation in the wild, properties of both medical images themselves and of existing medical datasets hinder training deeper and larger models because of overfitting. To this end, we propose a novel two-stream UNET architecture for automatic end-to-end medical image segmentation, in which intensity value and gradient vector flow (GVF) are two inputs for each stream, respectively. We demonstrate that two-stream CNNs with more low-level features gre…
In this paper, we propose a novel medical image segmentation using iterative deep learning framework. We have combined an iterative learning approach and an encoder-decoder network to improve segmentation results, which enables to precisely localize the regions of interest (ROIs) including complex shapes or detailed textures of medical images in an iterative manner. The proposed iterative deep convolutional encoder-decoder network consists of two main paths: convolutional encoder path and convolutional decoder path with iterative learning. Experimental results show that the proposed iterative …
Imperial College London（帝国理工学院）；BioMedIA Group, Department of Computing（计算系BioMedIA小组）
Few-shot semantic segmentation (FSS) has great potential for medical imaging applications. Most of the existing FSS techniques require abundant annotated semantic classes for training. However, these methods may not be applicable for medical images due to the lack of annotations. To address this problem we make several contributions: (1) A novel self-supervised FSS framework for medical images in order to eliminate the requirement for annotations during training. Additionally, superpixel-based pseudo-labels are generated to provide supervision; (2) An adaptive local prototype pooling module pl…
Domain adaptation for semantic image segmentation is very necessary since manually labeling large datasets with pixel-level labels is expensive and time consuming. Existing domain adaptation techniques either work on limited datasets, or yield not so good performance compared with supervised learning. In this paper, we propose a novel bidirectional learning framework for domain adaptation of segmentation. Using the bidirectional learning, the image translation model and the segmentation adaptation model can be learned alternatively and promote to each other. Furthermore, we propose a self-supe…
We propose an attention mechanism for 3D medical image segmentation. The method, named segmentation-by-detection, is a cascade of a detection module followed by a segmentation module. The detection module enables a region of interest to come to attention and produces a set of object region candidates which are further used as an attention model. Rather than dealing with the entire volume, the segmentation module distills the information from the potential region. This scheme is an efficient solution for volumetric data as it reduces the influence of the surrounding noise which is especially im…
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
University of Oxford（牛津大学）；National University of Singapore（新加坡国立大学）；University of Alberta（阿尔伯塔大学）；A*STAR（新加坡科技研究局）；Carnegie Mellon University（卡内基梅隆大学）；Singapore Eye Research Institute（新加坡眼科研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Hongkong（香港大学）
本文提出Medical SAM Adapter（Med-SA），通过轻量适配技术将医学知识融入SAM，利用SD-Trans和HyP-Adpt适配2D模型至3D医学图像，在17个任务上以仅2%参数更新超越SOTA方法。
Université de technologie de Compiègne（贡比涅技术大学）；University of Rouen Normandy（诺曼底鲁昂大学）；Institut universitaire de France（法国大学研究院）
The investigation of uncertainty is of major importance in risk-critical applications, such as medical image segmentation. Belief function theory, a formal framework for uncertainty analysis and multiple evidence fusion, has made significant contributions to medical image segmentation, especially since the development of deep learning. In this paper, we provide an introduction to the topic of medical image segmentation methods using belief function theory. We classify the methods according to the fusion step and explain how information with uncertainty or imprecision is modeled and fused with …
SEG-SAM: Semantic-Guided SAM for Unified Medical Image Segmentation
School of Electronic and Information Engineering, South China University of Technology（华南理工大学电子与信息工程学院）；Pazhou Lab（琶洲实验室）；College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）
