尽管计算机视觉已被证实对医学图像分割具有重要价值，但其应用仍面临数据集规模有限、有效利用未标注图像复杂度高等挑战。为应对这些挑战，我们提出了一种全新的基于一致性的半监督方法，名为数据高效医学分割器（data-efficient medical segmenter，DEMS）。DEMS采用编码器-解码器架构，并整合了自研的在线自动增强器（online automatic augmenter，OAA）和残差鲁棒性增强（residual robustness enhancement，RRE）模块。OAA通过多种图像变换对输入数据进行增强，从而丰富数据集多样性以提升泛化能力。RRE丰富了特征多样性，并引入扰动为不同解码器生成多样化输入，进而增强可变性。此外，我们还引入了一种敏感损失函数，以进一步增强不同解码器之间的一致性并稳定训练过程。在自有数据集和三个公开数据集上开展的大量实验结果证实了DEMS的有效性。在极端数据短缺场景下，我们的DEMS在戴斯系数（dice score）上相比U-Net和性能最优的现有先进方法分别提升了16.85%和10.37%。凭借其优异的数据效率，DEMS有望为小数据场景下的医学分割领域带来重大进展。该项目主页可通过https://github.com/NUS-Tim/DEMS访问。
英文摘要：
While computer vision has proven valuable for medical image segmentation, its application faces challenges such as limited dataset sizes and the complexity of effectively leveraging unlabeled images. To address these challenges, we present a novel semi-supervised, consistency-based approach termed the data-efficient medical segmenter (DEMS). The DEMS features an encoder-decoder architecture and incorporates the developed online automatic augmenter (OAA) and residual robustness enhancement (RRE) blocks. The OAA augments input data with various image transformations, thereby diversifying the dataset to improve the generalization ability. The RRE enriches feature diversity and introduces perturbations to create varied inputs for different decoders, thereby providing enhanced variability. Moreover, we introduce a sensitive loss to further enhance consistency across different decoders and stabilize the training process. Extensive experimental results on both our own and three public datasets affirm the effectiveness of DEMS. Under extreme data shortage scenarios, our DEMS achieves 16.85\% and 10.37\% improvement in dice score compared with the U-Net and top-performed state-of-the-art method, respectively. Given its superior data efficiency, DEMS could present significant advancements in medical segmentation under small data regimes. The project homepage can be accessed at https://github.com/NUS-Tim/DEMS.
East China Normal University（华东师范大学）；Tongji University（同济大学）
The encoder-decoder networks are commonly used in medical image segmentation due to their remarkable performance in hierarchical feature fusion. However, the expanding path for feature decoding and spatial recovery does not consider the long-term dependency when fusing feature maps from different layers, and the universal encoder-decoder network does not make full use of the multi-modality information to improve the network robustness especially for segmenting medical MRI. In this paper, we propose a novel feature fusion unit called Recurrent Decoding Cell (RDC) which leverages convolutional R…
Exploiting Scale-Variant Attention for Segmenting Small Medical Objects
Centre for Robotics and Automation, City University of Hong Kong（香港城市大学机器人与自动化中心）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）
University of Hamburg（汉堡大学）；University Medical Center Hamburg-Eppendorf（汉堡-埃彭多夫大学医学中心）
Precisely locating and segmenting medical instruments in images of minimally invasive surgeries, medical instrument segmentation, is an essential first step for several tasks in medical image processing. However, image degradations, small instruments, and the generalization between different surgery types make medical instrument segmentation challenging. To cope with these challenges, we adapt the object proposal generation system AttentionMask and propose a dedicated post-processing to select promising proposals. The results on the ROBUST-MIS Challenge 2019 show that our adapted AttentionMask…
Latents of latents to delineate pixels: hybrid Matryoshka autoencoder-to-U-Net pairing for segmenting large medical images in GPU-poor and low-data regimes
School of Mathematics and Computer Science, Institute of Business Administration（商业管理学院数学与计算机科学学院）
Skip and Skip: Segmenting Medical Images with Prompts
Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；Engineering Research Center of AI and Robotics, Ministry of Education, China（教育部人工智能与机器人工程研究中心）；Jilin Provincial Key Laboratory of Intelligence Science（吉林省智能科学重点实验室）
J. J. Strossmayer University of Osijek（奥西耶克J.J.斯特罗斯马耶大学）；Ghent University（根特大学）；imec（比利时微电子研究中心）
Atrial fibrillation (AF) is a cardiovascular disease identified as one of the main risk factors for stroke. The majority of strokes due to AF are caused by clots originating in the left atrial appendage (LAA). LAA occlusion is an effective procedure for reducing stroke risk. Planning the procedure using pre-procedural imaging and analysis has shown benefits. The analysis is commonly done by manually segmenting the appendage on 2D slices. Automatic LAA segmentation methods could save an expert's time and provide insightful 3D visualizations and accurate automatic measurements to aid in medical …
A Novel Momentum-Based Deep Learning Techniques for Medical Image Classification and Segmentation
Northwestern University（西北大学）；IIIT Delhi（德里印度信息技术学院）；NSIT Delhi（德里国家信息技术学院）；Space Research Institute (IWF) of Austrian Academy of Sciences（奥地利科学院空间研究所）
We explore encoding brain symmetry into a neural network for a brain tumor segmentation task. A healthy human brain is symmetric at a high level of abstraction, and the high-level asymmetric parts are more likely to be tumor regions. Paying more attention to asymmetries has the potential to boost the performance in brain tumor segmentation. We propose a method to encode brain symmetry into existing neural networks and apply the method to a state-of-the-art neural network for medical imaging segmentation. We evaluate our symmetry-encoded network on the dataset from a brain tumor segmentation ch…
PAM: A Propagation-Based Model for Segmenting Any 3D Objects across Multi-Modal Medical Images
National Engineering Laboratory for Big Data Analysis and Applications, Peking University（北京大学国家大数据分析与应用工程实验室）；Beijing International Center for Mathematical Research (BICMR), Peking University（北京国际数学研究中心（BICMR），北京大学）；Center for Data Science, Peking University（北京大学数据科学中心）；Peking University Cancer Hospital and Institute（北京大学肿瘤医院暨肿瘤研究所）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）
École de Technologie Supérieure（高等技术学院）；Université de Sherbrooke（舍布鲁克大学）
This paper presents a client/server privacy-preserving network in the context of multicentric medical image analysis. Our approach is based on adversarial learning which encodes images to obfuscate the patient identity while preserving enough information for a target task. Our novel architecture is composed of three components: 1) an encoder network which removes identity-specific features from input medical images, 2) a discriminator network that attempts to identify the subject from the encoded images, 3) a medical image analysis network which analyzes the content of the encoded images (segm…
École de technologie Superieure (ETS)（高等技术学院（ETS））；IIT Kharagpur（印度理工学院卡拉格布尔分校）
We address the problem of segmenting 3D multi-modal medical images in scenarios where very few labeled examples are available for training. Leveraging the recent success of adversarial learning for semi-supervised segmentation, we propose a novel method based on Generative Adversarial Networks (GANs) to train a segmentation model with both labeled and unlabeled images. The proposed method prevents over-fitting by learning to discriminate between true and fake patches obtained by a generator network. Our work extends current adversarial learning approaches, which focus on 2D single-modality ima…
RadSAM: Segmenting 3D radiological images with a 2D promptable model
Raidium；Université de Paris Cité（巴黎大学）；AP-HP（法国国家卫生与医学研究院）；Hôpital Européen Georges Pompidou（欧仁·蓬皮杜医院）；Department of Vascular and Oncological Interventional Radiology（血管和肿瘤介入放射科）；HEKA INRIA；INSERM PARCC U 970（法国国家医学研究院 PARCC U 970）
Vanderbilt University（范德堡大学）；The George Washington University（乔治华盛顿大学）；Korea University（高丽大学）
Segmenting medical images accurately and reliably is important for disease diagnosis and treatment. It is a challenging task because of the wide variety of objects' sizes, shapes, and scanning modalities. Recently, many convolutional neural networks (CNN) have been designed for segmentation tasks and achieved great success. Few studies, however, have fully considered the sizes of objects, and thus most demonstrate poor performance for small objects segmentation. This can have a significant impact on the early detection of diseases. This paper proposes a Context Axial Reserve Attention Network …
Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation
Université de technologie de Compiègne（贡比涅技术大学）；CNRS（法国国家科学研究中心）；Université de Rouen Normandie（鲁昂诺曼底大学）；Centre Henri Becquerel（亨利·贝克勒尔中心）；Institut universitaire de France（法国大学研究院）
Vanderbilt University（范德比尔特大学）；The George Washington University（乔治华盛顿大学）
Segmenting medical images accurately and reliably is important for disease diagnosis and treatment. It is a challenging task because of the wide variety of objects' sizes, shapes, and scanning modalities. Recently, many convolutional neural networks (CNN) have been designed for segmentation tasks and achieved great success. Few studies, however, have fully considered the sizes of objects, and thus most demonstrate poor performance for small objects segmentation. This can have a significant impact on the early detection of diseases. This paper proposes a Context Axial Reverse Attention Network …
The George Washington University Medical Center（乔治华盛顿大学医学中心）
Recently, deep learning has become much more popular in computer vision area. The Convolution Neural Network (CNN) has brought a breakthrough in images segmentation areas, especially, for medical images. In this regard, U-Net is the predominant approach to medical image segmentation task. The U-Net not only performs well in segmenting multimodal medical images generally, but also in some tough cases of them. However, we found that the classical U-Net architecture has limitation in several aspects. Therefore, we applied modifications: 1) designed efficient CNN architecture to replace encoder an…
SAM3D: Segment Anything Model in Volumetric Medical Images
University of Arkansas（阿肯色大学）；University of Science, Vietnam National University（越南国家大学科学大学）；John von Neumann Institute, Vietnam National University（越南国家大学约翰·冯·诺依曼研究所）；West Virginia University（西弗吉尼亚大学）；University of Arkansas for Medical Sciences（阿肯色医科大学）
University of Science and Technology Beijing（北京科技大学）；University of Central Florida（中佛罗里达大学）
For 3D medical image (e.g. CT and MRI) segmentation, the difficulty of segmenting each slice in a clinical case varies greatly. Previous research on volumetric medical image segmentation in a slice-by-slice manner conventionally use the identical 2D deep neural network to segment all the slices of the same case, ignoring the data heterogeneity among image slices. In this paper, we focus on multi-modal 3D MRI brain tumor segmentation and propose a dynamic architecture network named Med-DANet based on adaptive model selection to achieve effective accuracy and efficiency trade-off. For each slice…
Enhancing Medical Imaging with GANs Synthesizing Realistic Images from Limited Data
Columbia University（哥伦比亚大学）；Texas Tech University（德克萨斯理工大学）；University of California, Davis（加利福尼亚大学戴维斯分校）；Johns Hopkins University（约翰斯·霍普金斯大学）；North Carolina State University（北卡罗来纳州立大学）
Philips Research（飞利浦研究院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Philips GmbH Innovative Technologies（飞利浦创新技术有限公司）
Chest X-ray is one of the most widespread examinations of the human body. In interventional radiology, its use is frequently associated with the need to visualize various tube-like objects, such as puncture needles, guiding sheaths, wires, and catheters. Detection and precise localization of these tube-like objects in the X-ray images is, therefore, of utmost value, catalyzing the development of accurate target-specific segmentation algorithms. Similar to the other medical imaging tasks, the manual pixel-wise annotation of the tubes is a resource-consuming process. In this work, we aim to alle…
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）
Deep convolutional neural networks (CNNs) have been widely used for medical image segmentation. In most studies, only the output layer is exploited to compute the final segmentation results and the hidden representations of the deep learned features have not been well understood. In this paper, we propose a prototype segmentation (ProtoSeg) method to compute a binary segmentation map based on deep features. We measure the segmentation abilities of the features by computing the Dice between the feature segmentation map and ground-truth, named as the segmentation ability score (SA score for shor…
The recent achievements of Deep Learning rely on the test data being similar in distribution to the training data. In an ideal case, Deep Learning models would achieve Out-of-Distribution (OoD) Generalization, i.e. reliably make predictions on out-of-distribution data. Yet in practice, models usually fail to generalize well when facing a shift in distribution. Several methods were thereby designed to improve the robustness of the features learned by a model through Regularization- or Domain-Prediction-based schemes. Segmenting medical images such as MRIs of the hippocampus is essential for the…
National Institutes of Health（美国国立卫生研究院）；University of Wisconsin（威斯康星大学）
Current deep learning based segmentation models often generalize poorly between domains due to insufficient training data. In real-world clinical applications, cross-domain image analysis tools are in high demand since medical images from different domains are often needed to achieve a precise diagnosis. An important example in radiology is generalizing from non-contrast CT to contrast enhanced CTs. Contrast enhanced CT scans at different phases are used to enhance certain pathologies or organs. Many existing cross-domain image-to-image translation models have been shown to improve cross-domai…
Semi-Supervised Disease Classification based on Limited Medical Image Data
School of Science, Minzu University of China（中央民族大学理学院）；Joint Research Center on Computational Mathematics and Control, Shenzhen MSU-BIT University（深圳北理莫斯科大学计算数学与控制联合研究中心）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究院）
Empowering Medical Imaging with Artificial Intelligence: A Review of Machine Learning Approaches for the Detection, and Segmentation of COVID-19 Using Radiographic and Tomographic Images
Isfahan University of Technology（伊斯法罕理工大学）；University of Michigan（密歇根大学）
Medical image analysis, especially segmenting a specific organ, has an important role in developing clinical decision support systems. In cardiac magnetic resonance (MR) imaging, segmenting the left and right ventricles helps physicians diagnose different heart abnormalities. There are challenges for this task, including the intensity and shape similarity between left ventricle and other organs, inaccurate boundaries and presence of noise in most of the images. In this paper we propose an automated method for segmenting the left ventricle in cardiac MR images. We first automatically extract th…
Despite advances in deep learning, robustness under domain shift remains a major bottleneck in medical imaging settings. Findings on natural images suggest that deep neural models can show a strong textural bias when carrying out image classification tasks. In this thorough empirical study, we draw inspiration from findings on natural images and investigate ways in which addressing the textural bias phenomenon could bring up the robustness of deep segmentation models when applied to three-dimensional (3D) medical data. To achieve this, publicly available MRI scans from the Developing Human Con…
Image segmentation is an important task in many medical applications. Methods based on convolutional neural networks attain state-of-the-art accuracy; however, they typically rely on supervised training with large labeled datasets. Labeling medical images requires significant expertise and time, and typical hand-tuned approaches for data augmentation fail to capture the complex variations in such images. We present an automated data augmentation method for synthesizing labeled medical images. We demonstrate our method on the task of segmenting magnetic resonance imaging (MRI) brain scans. Our …
Johns Hopkins University（约翰斯·霍普金斯大学）；Sigma Technologies（12西格玛科技）；Johns Hopkins School of Medicine（约翰斯·霍普金斯医学院）
Medical images are often used to detect and characterize pathology and disease; however, automatically identifying and segmenting pathology in medical images is challenging because the appearance of pathology across diseases varies widely. To address this challenge, we propose a Bayesian deep learning method that learns to translate healthy computed tomography images to magnetic resonance images and simultaneously calculates voxel-wise uncertainty. Since high uncertainty occurs in pathological regions of the image, this uncertainty can be used for unsupervised anomaly segmentation. We show enc…
Zhengzhou University（郑州大学）；Henan Institute of Advanced Technology（河南先进技术研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Suining Central Hospital（遂宁市中心医院）；Hwa Mei Hospital, University of Chinese Academy of Sciences（中国科学院大学华美医院）；The University of Sydney（悉尼大学）；NanChang University（南昌大学）；University of Electronic Science and Technology of China（电子科技大学）
The convolutional-based methods provide good segmentation performance in the medical image segmentation task. However, those methods have the following challenges when dealing with the edges of the medical images: (1) Previous convolutional-based methods do not focus on the boundary relationship between foreground and background around the segmentation edge, which leads to the degradation of segmentation performance when the edge changes complexly. (2) The inductive bias of the convolutional layer cannot be adapted to complex edge changes and the aggregation of multiple-segmented areas, result…
SegBook: A Simple Baseline and Cookbook for Volumetric Medical Image Segmentation
Shanghai AI Laboratory（上海人工智能实验室）；Xiamen University（厦门大学）；East China Normal University（华东师范大学）；University of Cambridge（剑桥大学）；Stanford University（斯坦福大学）
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）
The goal of this paper is to interactively refine the automatic segmentation on challenging structures that fall behind human performance, either due to the scarcity of available annotations or the difficulty nature of the problem itself, for example, on segmenting cancer or small organs. Specifically, we propose a novel Transformer-based architecture for Interactive Segmentation (TIS), that treats the refinement task as a procedure for grouping pixels with similar features to those clicks given by the end users. Our proposed architecture is composed of Transformer Decoder variants, which natu…
National Library of Medicine, National Institutes of Health（美国国立医学图书馆，美国国立卫生研究院）
Deep learning (DL) models are state-of-the-art in segmenting anatomical and disease regions of interest (ROIs) in medical images. Particularly, a large number of DL-based techniques have been reported using chest X-rays (CXRs). However, these models are reportedly trained on reduced image resolutions for reasons related to the lack of computational resources. Literature is sparse in discussing the optimal image resolution to train these models for segmenting the Tuberculosis (TB)-consistent lesions in CXRs. In this study, we investigated the performance variations using an Inception-V3 UNet mo…
The University of Tokyo（东京大学）；The University of British Columbia（不列颠哥伦比亚大学）；RIKEN, AIP（理化学研究所革新智能综合研究中心）；University of Texas Arlington（德克萨斯大学阿灵顿分校）
The human gaze is a cost-efficient physiological data that reveals human underlying attentional patterns. The selective attention mechanism helps the cognition system focus on task-relevant visual clues by ignoring the presence of distractors. Thanks to this ability, human beings can efficiently learn from a very limited number of training samples. Inspired by this mechanism, we aim to leverage gaze for medical image analysis tasks with small training data. Our proposed framework includes a backbone encoder and a Selective Attention Network (SAN) that simulates the underlying attention. The SA…
LIPN-UMR-CNRS 7030（法国国家科学研究中心7030联合研究单元LIPN实验室）；Sorbonne University Paris Nord（巴黎北索邦大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）；Laboratory of Images, Signals and Intelligent Systems（图像、信号与智能系统实验室）；Université Paris-Est Créteil（巴黎东克雷泰伊大学）
Manual segmentation of medical images (e.g., segmenting tumors in CT scans) is a high-effort task that can be accelerated with machine learning techniques. However, selecting the right segmentation approach depends on the evaluation function, particularly in medical image segmentation where we must deal with dependency between voxels. For instance, in contrast to classical systems where the predictions are either correct or incorrect, predictions in medical image segmentation may be partially correct and incorrect simultaneously. In this paper, we explore this expressiveness to extract the use…
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Cardiovascular Research Centre, Royal Brompton Hospital（皇家布朗普顿医院心血管研究中心）
Medical image classification and segmentation based on deep learning (DL) are emergency research topics for diagnosing variant viruses of the current COVID-19 situation. In COVID-19 computed tomography (CT) images of the lungs, ground glass turbidity is the most common finding that requires specialist diagnosis. Based on this situation, some researchers propose the relevant DL models which can replace professional diagnostic specialists in clinics when lacking expertise. However, although DL methods have a stunning performance in medical image processing, the limited datasets can be a challeng…
University of Nottingham Ningbo China（宁波诺丁汉大学）；School of Computer Science（计算机学院）
Segmentation of objects with various sizes is relatively less explored in medical imaging, and has been very challenging in computer vision tasks in general. We hypothesize that the receptive field of a deep model corresponds closely to the size of object to be segmented, which could critically influence the segmentation accuracy of objects with varied sizes. In this study, we employed "AmygNet", a dual-branch fully convolutional neural network (FCNN) with two different sizes of receptive fields, to investigate the effects of receptive field on segmenting four major subnuclei of bilateral amyg…
Local discriminative representation is needed in many medical image analysis tasks such as identifying sub-types of lesion or segmenting detailed components of anatomical structures. However, the commonly applied supervised representation learning methods require a large amount of annotated data, and unsupervised discriminative representation learning distinguishes different images by learning a global feature, both of which are not suitable for localized medical image analysis tasks. In order to avoid the limitations of these two methods, we introduce local discrimination into unsupervised re…
