Medical image segmentation, which aims to automatically extract anatomical or pathological structures, plays a key role in computer-aided diagnosis and disease analysis. Despite the problem has been widely studied, existing methods are prone to topological errors. In medical imaging, the topology of the structure, such as the kidney or lung, is usually known. Preserving the topology of the structure in the segmentation process is of utmost importance for accurate image analysis. In this work, a novel learning-based segmentation model is proposed. A {\it topology-preserving segmentation network (TPSN)} is trained to give an accurate segmentation result of an input image that preserves the prescribed topology. TPSN is a deformation-based model that yields a deformation map through a UNet, which takes the medical image and a template mask as inputs. The main idea is to deform a template mask describing the prescribed topology by a diffeomorphism to segment the object in the image. The topology of the shape in the template mask is well preserved under the diffeomorphic map. The diffeomorphic property of the map is controlled by introducing a regularization term related to the Jacobian in the loss function. As such, a topology-preserving segmentation result can be guaranteed. Furthermore, a multi-scale TPSN is developed in this paper that incorporates multi-level information of images to produce more precise segmentation results. To evaluate our method, we applied the 2D TPSN on Ham10000 and 3D TPSN on KiTS21. Experimental results illustrate our method outperforms the baseline UNet segmentation model with/without connected-component analysis (CCA) by both the dice score and IoU score. Besides, results show that our method can produce reliable results even in challenging cases, where pixel-wise segmentation models by UNet and CCA fail to obtain accurate results.
Gland segmentation is a critical step to quantitatively assess the morphology of glands in histopathology image analysis. However, it is challenging to separate densely clustered glands accurately. Existing deep learning-based approaches attempted to use contour-based techniques to alleviate this issue but only achieved limited success. To address this challenge, we propose a novel topology-aware network (TA-Net) to accurately separate densely clustered and severely deformed glands. The proposed TA-Net has a multitask learning architecture and enhances the generalization of gland segmentation …
Stony Brook University（石溪大学）；Graduate Center, CUNY（纽约市立大学研究生中心）
Structural accuracy of segmentation is important for finescale structures in biomedical images. We propose a novel TopologyAttention ConvLSTM Network (TACNet) for 3D image segmentation in order to achieve high structural accuracy for 3D segmentation tasks. Specifically, we propose a Spatial Topology-Attention (STA) module to process a 3D image as a stack of 2D image slices and adopt ConvLSTM to leverage contextual structure information from adjacent slices. In order to effectively transfer topology-critical information across slices, we propose an Iterative-Topology Attention (ITA) module that…
Universal Topology Refinement for Medical Image Segmentation with Polynomial Feature Synthesis
Imperial College London（帝国理工学院）；University of Oxford（牛津大学）；Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学右侧医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
Topology Optimization in Medical Image Segmentation with Fast Euler Characteristic
Department of Computing, Imperial College London（帝国理工学院计算系）；University of Oxford（牛津大学）；Weill Cornell Medicine（韦尔医学院）；Cornell Tech（康奈尔科技）；TUM University Hospital, Technical University of Munich（慕尼黑技术大学医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔兰根-纽伦堡弗里德里希-亚历山大大学）；King’s College London（伦敦国王学院）
Image segmentation is to extract meaningful objects from a given image. For degraded images due to occlusions, obscurities or noises, the accuracy of the segmentation result can be severely affected. To alleviate this problem, prior information about the target object is usually introduced. In [10], a topology-preserving registration-based segmentation model was proposed, which is restricted to segment 2D images only. In this paper, we propose a novel 3D topology-preserving registration-based segmentation model with the hyperelastic regularization, which can handle both 2D and 3D images. The e…
Recently, neural architecture search (NAS) has been applied to automatically search high-performance networks for medical image segmentation. The NAS search space usually contains a network topology level (controlling connections among cells with different spatial scales) and a cell level (operations within each cell). Existing methods either require long searching time for large-scale 3D image datasets, or are limited to pre-defined topologies (such as U-shaped or single-path). In this work, we focus on three important aspects of NAS in 3D medical image segmentation: flexible multi-path netwo…
Beijing Normal University（北京师范大学）；Hong Kong Baptist University（香港浸会大学）
Image segmentation with a volume constraint is an important prior for many real applications. In this work, we present a novel volume preserving image segmentation algorithm, which is based on the framework of entropic regularized optimal transport theory. The classical Total Variation (TV) regularizer and volume preserving are integrated into a regularized optimal transport model, and the volume and classification constraints can be regarded as two measures preserving constraints in the optimal transport problem. By studying the dual problem, we develop a simple and efficient dual algorithm f…
Stony Brook University（石溪大学）；Oregon State University（俄勒冈州立大学）
Segmentation algorithms are prone to make topological errors on fine-scale structures, e.g., broken connections. We propose a novel method that learns to segment with correct topology. In particular, we design a continuous-valued loss function that enforces a segmentation to have the same topology as the ground truth, i.e., having the same Betti number. The proposed topology-preserving loss function is differentiable and we incorporate it into end-to-end training of a deep neural network. Our method achieves much better performance on the Betti number error, which directly accounts for the top…
Capturing the global topology of an image is essential for proposing an accurate segmentation of its domain. However, most of existing segmentation methods do not preserve the initial topology of the given input, which is detrimental for numerous downstream object-based tasks. This is all the more true for deep learning models which most work at local scales. In this paper, we propose a new topology-preserving deep image segmentation method which relies on a new leakage loss: the Pathloss. Our method is an extension of the BALoss [1], in which we want to improve the leakage detection for bette…
We introduce a method for training neural networks to perform image or volume segmentation in which prior knowledge about the topology of the segmented object can be explicitly provided and then incorporated into the training process. By using the differentiable properties of persistent homology, a concept used in topological data analysis, we can specify the desired topology of segmented objects in terms of their Betti numbers and then drive the proposed segmentations to contain the specified topological features. Importantly this process does not require any ground-truth labels, just prior k…
National Pingtung University of Science and Technology（屏东科技大学）
Semantic segmentation is one of the most attractive research fields in computer vision. In the VIPriors challenge, only very limited numbers of training samples are allowed, leading to that the current state-of-the-art and deep learning-based semantic segmentation techniques are hard to train well. To overcome this shortcoming, therefore, we propose edge-preserving guidance to obtain the extra prior information, to avoid the overfitting under small-scale training dataset. First, a two-channeled convolutional layer is concatenated to the last layer of the conventional semantic segmentation netw…
Besides per-pixel accuracy, topological correctness is also crucial for the segmentation of images with fine-scale structures, e.g., satellite images and biomedical images. In this paper, by leveraging the theory of digital topology, we identify pixels in an image that are critical for topology. By focusing on these critical pixels, we propose a new homotopy warping loss to train deep image segmentation networks for better topological accuracy. To efficiently identify these topologically critical pixels, we propose a new algorithm exploiting the distance transform. The proposed algorithm, as w…
Indian Institute of Technology（印度理工学院）；Purdue University（普渡大学）；All India Institute of Medical Sciences（全印度医学科学研究所）
We investigate the applicability of U-Net based models for segmenting Urinary Bladder (UB) in male pelvic view UltraSound (US) images. The segmentation of UB in the US image aids radiologists in diagnosing the UB. However, UB in US images has arbitrary shapes, indistinct boundaries and considerably large inter- and intra-subject variability, making segmentation a quite challenging task. Our study of the state-of-the-art (SOTA) segmentation network, U-Net, for the problem reveals that it often fails to capture the salient characteristics of UB due to the varying shape and scales of anatomy in t…
TopoMortar: A dataset to evaluate image segmentation methods focused on topology accuracy
Department of Applied Mathematics and Computer Science, Technical University of Denmark（技术大学丹麦应用数学和计算机科学系）；A.I. Virtanen Institute, University of Eastern Finland（东部芬兰大学A.I. Virtanen研究所）；Department of Architecture, Faculty of Engineering, Sojo University, Japan（日本早稻田大学工学系建筑系）
Disconnect to Connect: A Data Augmentation Method for Improving Topology Accuracy in Image Segmentation
DTU Compute, Technical University of Denmark（丹麦技术大学DTU计算学院）；A.I. Virtanen Institute, University of Eastern Finland（东芬兰大学人工智能维尔塔宁研究所）；Aarhus University（奥胡斯大学）
Stony Brook University（石溪大学）；University of California, San Diego（加利福尼亚大学圣迭戈分校）；Oregon State University（俄勒冈州立大学）
In the segmentation of fine-scale structures from natural and biomedical images, per-pixel accuracy is not the only metric of concern. Topological correctness, such as vessel connectivity and membrane closure, is crucial for downstream analysis tasks. In this paper, we propose a new approach to train deep image segmentation networks for better topological accuracy. In particular, leveraging the power of discrete Morse theory (DMT), we identify global structures, including 1D skeletons and 2D patches, which are important for topological accuracy. Trained with a novel loss based on these global …
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM)（慕尼黑工业大学医疗保健与医学人工智能教席）；TUM University Hospital（慕尼黑工业大学医院）；School of Computation, Information and Technology, TUM（慕尼黑工业大学计算、信息与技术学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）；Weill Cornell Medicine, Cornell University（康奈尔大学威尔医学院）
Topology-Preserving Image Segmentation with Spatial-Aware Persistent Feature Matching
Department of Electrical and Computer Engineering, University of California, San Diego（电气与计算机工程系，加州大学圣地亚哥分校）；Department of Ophthalmology, University of California, San Diego（眼科学系，加州大学圣地亚哥分校）
G-images refer to image data defined on irregular graph domains. This work elaborates a similarity-preserving Fuzzy C-Means (FCM) algorithm for G-image segmentation and aims to develop techniques and tools for segmenting G-images. To preserve the membership similarity between an arbitrary image pixel and its neighbors, a Kullback-Leibler divergence term on membership partition is introduced as a part of FCM. As a result, similarity-preserving FCM is developed by considering spatial information of image pixels for its robustness enhancement. Due to superior characteristics of a wavelet space, t…
University of Leeds（利兹大学）；Leeds Institute for Cardiovascular and Metabolic Medicine（利兹心血管与代谢医学研究所）；KU Leuven（荷语鲁汶大学）；Alan Turing Institute（艾伦·图灵研究所）
Medical image registration is a challenging task involving the estimation of spatial transformations to establish anatomical correspondence between pairs or groups of images. Recently, deep learning-based image registration methods have been widely explored, and demonstrated to enable fast and accurate image registration in a variety of applications. However, most deep learning-based registration methods assume that the deformation fields are smooth and continuous everywhere in the image domain, which is not always true, especially when registering images whose fields of view contain discontin…
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University（北京交通大学先进信息科学与网络技术北京市重点实验室）；Department of Computer Science and Engineering, Michigan State University（密歇根州立大学计算机科学与工程系）
To bridge the gap between the source and target domains in unsupervised domain adaptation (UDA), the most common strategy puts focus on matching the marginal distributions in the feature space through adversarial learning. However, such category-agnostic global alignment lacks of exploiting the class-level joint distributions, causing the aligned distribution less discriminative. To address this issue, we propose in this paper a novel margin preserving self-paced contrastive Learning (MPSCL) model for cross-modal medical image segmentation. Unlike the conventional construction of contrastive p…
DTEA: Dynamic Topology Weaving and Instability-Driven Entropic Attenuation for Medical Image Segmentation
School of Advanced Manufacturing, Guangdong University of Technology（广东技术大学先进制造学院）；School of Electronic and Communication Engineering, Shenzhen Polytechnic University（深圳职业技术大学电子与通信工程学院）；Guangzhou City University of Technology（广州市城市技术大学）；Faculty of Science and Technology, University of Macau（澳门大学科学与技术学院）；Department of Cardiology, Guangdong Provincial People’s Hospital (Guangdong Academy of Medical Sciences), Southern Medical University（广东省人民医院心内科（广东省医学科学院））；Guangdong Cardiovascular Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences（广东省心血管病研究所，广东省人民医院，广东省医学科学院）
Continuous-tone Simple Points: An $\ell_0$-Norm of Cyclic Gradient for Topology-Preserving Data-Driven Image Segmentation
Laboratory of Mathematics and Complex Systems (Ministry of Education), School of Mathematical Sciences, Beijing Normal University（数学与复杂系统实验室（教育部）, 数学科学学院, 北京师范大学）；State Key Laboratory of Remote Sensing Science, Faculty of Geographical Science, Beijing Normal University（遥感科学国家重点实验室, 地理科学学院, 北京师范大学）
Topograph: An efficient Graph-Based Framework for Strictly Topology Preserving Image Segmentation
School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Department of Computing, Imperial College London（帝国理工学院计算系）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute, Technical University of Munich（慕尼黑工业大学慕尼黑数据科学研究所）；Weill Cornell Medicine, Cornell University（康奈尔大学威尔医学院）
Privacy-Preserving Semantic Segmentation from High-Resolution Depth and Ultra-Low-Resolution RGB
University of Bonn（波恩大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习和人工智能研究所）；Center for Robotics（机器人中心）；Bonn-Rhein-Sieg University of Applied Sciences（波恩-莱茵-锡格应用科学大学）；Fraunhofer Institute for Intelligent Analysis and Information Systems（弗劳恩霍夫智能分析和信息系统研究所）
Depth-Only Open-Vocabulary 3D Semantic Segmentation For Privacy-Preserving Robotic Applications
Humanoid Robots Lab, University of Bonn（波恩大学仿人机器人实验室）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习和人工智能研究所）；Center for Robotics, Bonn, Germany（波恩机器人中心）
Topology-Guaranteed Image Segmentation: Enforcing Connectivity, Genus, and Width Constraints
Laboratory of Mathematics and Complex Systems (Ministry of Education of China)（数学与复杂系统实验室（教育部））；School of Mathematical Sciences, Beijing Normal University（北京师范大学数学系）；NORCE Norwegian Research Centre（挪威国家研究中心）；Beijing Normal University（北京师范大学）
Institute of Biomedical Engineering（生物医学工程研究所）；University of Oxford（牛津大学）；Wellcome Centre for Integrative Neuroimaging（惠康综合神经成像中心）；FMRIB（FMRIB（功能磁共振成像与脑研究中心））；Australian Institute for Machine Learning (AIML)（澳大利亚机器学习研究所（AIML））；University of Adelaide（阿德莱德大学）；South Australian Health and Medical Research Institute (SAHMRI)（南澳大利亚健康与医学研究所（SAHMRI））
Accurate topology is key when performing meaningful anatomical segmentations, however, it is often overlooked in traditional deep learning methods. In this work we propose TEDS-Net: a novel segmentation method that guarantees accurate topology. Our method is built upon a continuous diffeomorphic framework, which enforces topology preservation. However, in practice, diffeomorphic fields are represented using a finite number of parameters and sampled using methods such as linear interpolation, violating the theoretical guarantees. We therefore introduce additional modifications to more strictly …
Preserving Cardiac Integrity: A Topology-Infused Approach to Whole Heart Segmentation
Imperial College London（帝国理工学院）；Jilin university（吉林大学）；National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；School of Biomedical Engineering & Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）；Cardiovascular Research Centre, Royal Brompton Hospital（皇家布朗普顿医院心血管研究中心）
Costal Cartilage Segmentation with Topology Guided Deformable Mamba: Method and Benchmark
Plastic Surgery Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院整形外科医院）；School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Nanjing University of Science and Technology（南京理工大学）；The Second Hospital of Hebei Medical University（河北医科大学第二医院）
