AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
The University of Hong Kong（香港大学）；Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）深圳大数据研究院）；Longgang Central Hospital of Shenzhen（深圳龙岗中心医院）；Longgang People Hospital of Shenzhen（深圳龙岗人民医院）
Yuanfeng Ji, Haotian Bai, Jie Yang, Chongjian Ge, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhang, Wanling Ma, Xiang Wan, Ping Luo
Despite the considerable progress in automatic abdominal multi-organ segmentation from CT/MRI scans in recent years, a comprehensive evaluation of the models' capabilities is hampered by the lack of a large-scale benchmark from diverse clinical scenarios. Constraint by the high cost of collecting and labeling 3D medical data, most of the deep learning models to date are driven by datasets with a limited number of organs of interest or samples, which still limits the power of modern deep models and makes it difficult to provide a fully comprehensive and fair estimate of various methods. To mitigate the limitations, we present AMOS, a large-scale, diverse, clinical dataset for abdominal organ segmentation. AMOS provides 500 CT and 100 MRI scans collected from multi-center, multi-vendor, multi-modality, multi-phase, multi-disease patients, each with voxel-level annotations of 15 abdominal organs, providing challenging examples and test-bed for studying robust segmentation algorithms under diverse targets and scenarios. We further benchmark several state-of-the-art medical segmentation models to evaluate the status of the existing methods on this new challenging dataset. We have made our datasets, benchmark servers, and baselines publicly available, and hope to inspire future research. Information can be found at https://amos22.grand-challenge.org.
Muroran Institute of Technology（室兰工业大学）；Niigata University of Health and Welfare（新潟医疗福祉大学）
To promote the development of medical image segmentation technology, AMOS, a large-scale abdominal multi-organ dataset for versatile medical image segmentation, is provided and AMOS 2022 challenge is held by using the dataset. In this report, we present our solution for the AMOS 2022 challenge. We employ residual U-Net with deep super vision as our base model. The experimental results show that the mean scores of Dice similarity coefficient and normalized surface dice are 0.8504 and 0.8476 for CT only task and CT/MRI task, respectively.
Automated abdominal multi-organ segmentation is a crucial yet challenging task in the computer-aided diagnosis of abdominal organ-related diseases. Although numerous deep learning models have achieved remarkable success in many medical image segmentation tasks, accurate segmentation of abdominal organs remains challenging, due to the varying sizes of abdominal organs and the ambiguous boundaries among them. In this paper, we propose a boundary-aware network (BA-Net) to segment abdominal organs on CT scans and MRI scans. This model contains a shared encoder, a boundary decoder, and a segmentati…
MRISegmentator-Abdomen: A Fully Automated Multi-Organ and Structure Segmentation Tool for T1-weighted Abdominal MRI
Radiology and Imaging Sciences（放射学与影像科学）；National Institutes of Health Clinical Center（国家卫生研究院临床中心）；Walter Reed National Military Medical Center（沃尔特·里德国家军事医学中心）；Biostatistics and Clinical Epidemiology Services（生物统计与临床流行病学服务）
Devil is in Details: Locality-Aware 3D Abdominal CT Volume Generation for Self-Supervised Organ Segmentation
National Engineering Research Center for Multimedia Software（多媒体软件国家工程研究中心）；Hubei Key Laboratory of Multimedia and Network Communication Engineering（湖北省多媒体与网络通信工程重点实验室）；Institute of Artificial Intelligence（人工智能研究院）；School of Computer Science（计算机科学学院）
Large curated datasets are necessary, but annotating medical images is a time-consuming, laborious, and expensive process. Therefore, recent supervised methods are focusing on utilizing a large amount of unlabeled data. However, to do so, is a challenging task. To address this problem, we propose a new 3D Cross-Pseudo Supervision (3D-CPS) method, a semi-supervised network architecture based on nnU-Net with the Cross-Pseudo Supervision method. We design a new nnU-Net based preprocessing. In addition, we set the semi-supervised loss weights to expand linearity with each epoch to prevent the mode…
AbdomenGen: Sequential Volume-Conditioned Diffusion Framework for Abdominal Anatomy Generation
Center for Virtual Imaging Trials, Dept. of Radiology, Duke University School of Medicine（虚拟成像试验中心，放射科，杜克大学医学部）；Dept. of Electrical and Computer Engineering, Pratt School of Engineering, Duke University（电气与计算机工程系，普拉特工程学院，杜克大学）；Dept. of Physics, Trinity School of Arts and Sciences, Duke University（物理系，特里尼蒂艺术与科学学院，杜克大学）
AbdomenGen提出了一种序列体积条件扩散框架，通过Volume Control Scalar实现可控的腹部解剖生成，实现了高几何保真度和多器官解耦调节。
Nagoya University（名古屋大学）；Aichi Cancer Center Hospital（爱知县癌症中心医院）
Deep learning-based methods achieved impressive results for the segmentation of medical images. With the development of 3D fully convolutional networks (FCNs), it has become feasible to produce improved results for multi-organ segmentation of 3D computed tomography (CT) images. The results of multi-organ segmentation using deep learning-based methods not only depend on the choice of networks architecture, but also strongly rely on the choice of loss function. In this paper, we present a discussion on the influence of Dice-based loss functions for multi-class organ segmentation using a dataset …
LEAVS: An LLM-based Labeler for Abdominal CT Supervision
National Institutes of Health Clinical Center（美国国家卫生研究院临床中心）；Icahn School of Medicine at Mount Sinai（伊坎医学院）；Walter Reed National Military Medical Center（沃尔特·里德国家军事医疗中心）
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）
Organ segmentation is a prerequisite for a computer-aided diagnosis (CAD) system to detect pathologies and perform quantitative analysis. For anatomically high-variability abdominal organs such as the pancreas, previous segmentation works report low accuracies when comparing to organs like the heart or liver. In this paper, a fully-automated bottom-up method is presented for pancreas segmentation, using abdominal computed tomography (CT) scans. The method is based on a hierarchical two-tiered information propagation by classifying image patches. It labels superpixels as pancreas or not via poo…
A Multi-Center Benchmark for Abdominal Disease Diagnosis and Report Generation from Non-Contrast CT
Ain Shams University（艾因夏姆斯大学）；The Hong Kong University of Science and Technology（香港科技大学）；University of Catania（卡塔尼亚大学）；Universitat de Barcelona（巴塞罗那大学）
Vanderbilt University（范德堡大学）；Sigma Technology（12西格玛技术公司）；Vanderbilt University Medical Center（范德堡大学医学中心）
Abdominal multi-organ segmentation of computed tomography (CT) images has been the subject of extensive research interest. It presents a substantial challenge in medical image processing, as the shape and distribution of abdominal organs can vary greatly among the population and within an individual over time. While continuous integration of novel datasets into the training set provides potential for better segmentation performance, collection of data at scale is not only costly, but also impractical in some contexts. Moreover, it remains unclear what marginal value additional data have to off…
Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography
City University of Hong Kong（香港城市大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of California, San Francisco（加州大学旧金山分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Chinese University of Hong Kong（香港中文大学）；NVIDIA（英伟达）
The RSNA Abdominal Traumatic Injury CT (RATIC) Dataset
Scripps Clinic Medical Group（斯克里普斯诊所医疗集团）；University of California San Diego（加利福尼亚大学圣迭戈分校）；The Jackson Laboratory（杰克逊实验室）；Vancouver General Hospital（温哥华综合医院）；The Ohio State University（俄亥俄州立大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）；Thomas Jefferson University（托马斯·杰斐逊大学）；Duke University School of Medicine（杜克大学医学院）；Weill Cornell Medicine（威尔康奈尔医学院）；Yale University School of Medicine（耶鲁大学医学院）；University of California San Francisco（加利福尼亚大学旧金山分校）；University of California, Irvine（加利福尼亚大学尔湾分校）；Sunnybrook Health Sciences Centre（桑尼布鲁克健康科学中心）；University of Toronto（多伦多大学）；Alfred Health（阿尔弗雷德健康）；Monash University（莫纳什大学）；University of Utah（犹他大学）
The Johns Hopkins University（约翰斯·霍普金斯大学）；Shanghai University（上海大学）；The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
Deep neural networks have been widely adopted for automatic organ segmentation from abdominal CT scans. However, the segmentation accuracy of some small organs (e.g., the pancreas) is sometimes below satisfaction, arguably because deep networks are easily disrupted by the complex and variable background regions which occupies a large fraction of the input volume. In this paper, we formulate this problem into a fixed-point model which uses a predicted segmentation mask to shrink the input region. This is motivated by the fact that a smaller input region often leads to more accurate segmentation…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；University of Lübeck（吕贝克大学）
The construction of three-dimensional multi-modal tissue maps provides an opportunity to spur interdisciplinary innovations across temporal and spatial scales through information integration. While the preponderance of effort is allocated to the cellular level and explore the changes in cell interactions and organizations, contextualizing findings within organs and systems is essential to visualize and interpret higher resolution linkage across scales. There is a substantial normal variation of kidney morphometry and appearance across body size, sex, and imaging protocols in abdominal computed…
Graduate School of Informatics, Kyoto University（京都大学信息学研究科）；Graduate School of Medicine, Kyoto University（京都大学医学研究科）
Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a 3D organ mesh for a single-viewpoint 2D projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiati…
The Johns Hopkins University（约翰斯·霍普金斯大学）；The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
Automatic segmentation of an organ and its cystic region is a prerequisite of computer-aided diagnosis. In this paper, we focus on pancreatic cyst segmentation in abdominal CT scan. This task is important and very useful in clinical practice yet challenging due to the low contrast in boundary, the variability in location, shape and the different stages of the pancreatic cancer. Inspired by the high relevance between the location of a pancreas and its cystic region, we introduce extra deep supervision into the segmentation network, so that cyst segmentation can be improved with the help of rela…
Emory University（埃默里大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）
In this work, we propose an adversarial attack-based data augmentation method to improve the deep-learning-based segmentation algorithm for the delineation of Organs-At-Risk (OAR) in abdominal Computed Tomography (CT) to facilitate radiation therapy. We introduce Adversarial Feature Attack for Medical Image (AFA-MI) augmentation, which forces the segmentation network to learn out-of-distribution statistics and improve generalization and robustness to noises. AFA-MI augmentation consists of three steps: 1) generate adversarial noises by Fast Gradient Sign Method (FGSM) on the intermediate featu…
MRSegmentator: Multi-Modality Segmentation of 40 Classes in MRI and CT
Department of Radiology, Charité - Universitätsmedizin Berlin（柏林查理医院放射科）；Freie Universität Berlin（柏林自由大学）；Humboldt Universität zu Berlin（柏林洪堡大学）；Diagnostic Image Analysis Group, Radboud University Medical Center（拉德堡德大学医学中心诊断图像分析组）；School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital（医学与健康学院，莱比锡大学医院）；Institute of Radiology, University Hospital Erlangen, Friedrich-Alexander-Universität (FAU) Erlangen-Nürnberg（埃朗根大学医院放射科，埃朗根-纽伦堡弗里德里希-亚历山大大学）；Laboratory for Computer Aided Medical Procedures, Technical University of Munich（慕尼黑技术大学计算机辅助医疗程序实验室）；Chair for AI in Medicine and Healthcare, Klinikum rechts der Isar, Technical University Munich（医学与健康中的AI教授职位，莱比锡大学医院，慕尼黑技术大学）
Swinburne University of Technology（斯威本科技大学）；Victoria University（维多利亚大学）；Kyung Hee University（庆熙大学）
Quantitative assessment of the abdominal region from clinically acquired CT scans requires the simultaneous segmentation of abdominal organs. Thanks to the availability of high-performance computational resources, deep learning-based methods have resulted in state-of-the-art performance for the segmentation of 3D abdominal CT scans. However, the complex characterization of organs with fuzzy boundaries prevents the deep learning methods from accurately segmenting these anatomical organs. Specifically, the voxels on the boundary of organs are more vulnerable to misprediction due to the highly-va…
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
We developed a new joint probabilistic segmentation and image distribution matching generative adversarial network (PSIGAN) for unsupervised domain adaptation (UDA) and multi-organ segmentation from magnetic resonance (MRI) images. Our UDA approach models the co-dependency between images and their segmentation as a joint probability distribution using a new structure discriminator. The structure discriminator computes structure of interest focused adversarial loss by combining the generated pseudo MRI with probabilistic segmentations produced by a simultaneously trained segmentation sub-networ…
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
Vision transformers, with their ability to more efficiently model long-range context, have demonstrated impressive accuracy gains in several computer vision and medical image analysis tasks including segmentation. However, such methods need large labeled datasets for training, which is hard to obtain for medical image analysis. Self-supervised learning (SSL) has demonstrated success in medical image segmentation using convolutional networks. In this work, we developed a \underline{s}elf-distillation learning with \underline{m}asked \underline{i}mage modeling method to perform SSL for vision \u…
Objective : Abdominal anatomy segmentation is crucial for numerous applications from computer-assisted diagnosis to image-guided surgery. In this context, we address fully-automated multi-organ segmentation from abdominal CT and MR images using deep learning. Methods: The proposed model extends standard conditional generative adversarial networks. Additionally to the discriminator which enforces the model to create realistic organ delineations, it embeds cascaded partially pre-trained convolutional encoder-decoders as generator. Encoder fine-tuning from a large amount of non-medical images all…
Shape reconstruction of deformable organs from two-dimensional X-ray images is a key technology for image-guided intervention. In this paper, we propose an image-to-graph convolutional network (IGCN) for deformable shape reconstruction from a single-viewpoint projection image. The IGCN learns relationship between shape/deformation variability and the deep image features based on a deformation mapping scheme. In experiments targeted to the respiratory motion of abdominal organs, we confirmed the proposed framework with a regularized loss function can reconstruct liver shapes from a single digit…
A two-step concept for 3D segmentation on 5 abdominal organs inside volumetric CT images is presented. First each relevant organ's volume of interest is extracted as bounding box. The extracted volume acts as input for a second stage, wherein two compared U-Nets with different architectural dimensions re-construct an organ segmentation as label mask. In this work, we focus on comparing 2D U-Nets vs. 3D U-Net counterparts. Our initial results indicate Dice improvements of about 6\% at maximum. In this study to our surprise, liver and kidneys for instance were tackled significantly better using …
ORACLE-CT: Anatomy-Aware Support Pooling for CT Classification
Center for Virtual Imaging Trials, RAI Labs, Department of Radiology, Duke University（虚拟成像试验中心，RAI实验室，放射学系，杜克大学）；Electrical and Computer Engineering, Pratt School of Engineering, Duke University（电气与计算机工程，工程学院，杜克大学）；Department of Mathematics, Trinity College of Arts & Sciences, Duke University（数学系，艺术与科学学院，杜克大学）；Department of Radiology and Imaging Sciences, University of Arizona College of Medicine（放射学与影像科学系，亚利桑那大学医学院）
Accurate and robust segmentation of small organs in whole-body MRI is difficult due to anatomical variation and class imbalance. Recent deep network based approaches have demonstrated promising performance on abdominal multi-organ segmentations. However, the performance on small organs is still suboptimal as these occupy only small regions of the whole-body volumes with unclear boundaries and variable shapes. A coarse-to-fine, hierarchical strategy is a common approach to alleviate this problem, however, this might miss useful contextual information. We propose a two-stage approach with weight…
TotalSegmentator MRI: Robust Sequence-independent Segmentation of Multiple Anatomic Structures in MRI
University Hospital Basel（巴塞尔大学医院）；University of Freiburg（弗莱堡大学）；University Hospital of Tuebingen（蒂宾根大学医院）；German Cancer Research Center（德国癌症研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Human body is a complex dynamic system composed of various sub-dynamic parts. Especially, thoracic and abdominal organs have complex internal shape variations with different frequencies by various reasons such as respiration with fast motion and peristalsis with slower motion. CT protocols for abdominal lesions are multi-phase scans for various tumor detection to use different vascular contrast, however, they are not aligned well enough to visually check the same area. In this paper, we propose a time-efficient and accurate deformable registration algorithm for multi-phase CT scans considering…
Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Inpainting has recently been proposed as a successful deep learning technique for unsupervised medical image model discovery. The masks used for inpainting are generally independent of the dataset and are not tailored to perform on different given classes of anatomy. In this work, we introduce a method for generating shape-aware masks for inpainting, which aims at learning the statistical shape prior. We hypothesize that although the variation of masks improves the generalizability of inpainting models, the shape of the masks should follow the topology of the organs of interest. Hence, we prop…
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）
Automatic organ segmentation is an important yet challenging problem for medical image analysis. The pancreas is an abdominal organ with very high anatomical variability. This inhibits previous segmentation methods from achieving high accuracies, especially compared to other organs such as the liver, heart or kidneys. In this paper, we present a probabilistic bottom-up approach for pancreas segmentation in abdominal computed tomography (CT) scans, using multi-level deep convolutional networks (ConvNets). We propose and evaluate several variations of deep ConvNets in the context of hierarchical…
The Johns Hopkins University（约翰斯·霍普金斯大学）；Noah’s Ark Lab, Huawei Inc.（华为诺亚方舟实验室）；The Johns Hopkins Medical Institute（约翰斯·霍普金斯医学研究所）
There has been a debate in 3D medical image segmentation on whether to use 2D or 3D networks, where both pipelines have advantages and disadvantages. 2D methods enjoy a low inference time and greater transfer-ability while 3D methods are superior in performance for hard targets requiring contextual information. This paper investigates efficient 3D segmentation from another perspective, which uses 2D networks to mimic 3D segmentation. To compensate the lack of contextual information in 2D manner, we propose to thicken the 2D network inputs by feeding multiple slices as multiple channels into 2D…
Accurate automatic organ segmentation is an important yet challenging problem for medical image analysis. The pancreas is an abdominal organ with very high anatomical variability. This inhibits traditional segmentation methods from achieving high accuracies, especially compared to other organs such as the liver, heart or kidneys. In this paper, we present a holistic learning approach that integrates semantic mid-level cues of deeply-learned organ interior and boundary maps via robust spatial aggregation using random forest. Our method generates boundary preserving pixel-wise class labels for p…
Deformable registration of two-dimensional/three-dimensional (2D/3D) images of abdominal organs is a complicated task because the abdominal organs deform significantly and their contours are not detected in two-dimensional X-ray images. We propose a supervised deep learning framework that achieves 2D/3D deformable image registration between 3D volumes and single-viewpoint 2D projected images. The proposed method learns the translation from the target 2D projection images and the initial 3D volume to 3D displacement fields. In experiments, we registered 3D-computed tomography (CT) volumes to di…
Accurate and robust abdominal multi-organ segmentation from CT imaging of different modalities is a challenging task due to complex inter- and intra-organ shape and appearance variations among abdominal organs. In this paper, we propose a probabilistic multi-organ segmentation network with hierarchical spatial-wise feature modulation to capture flexible organ semantic variants and inject the learnt variants into different scales of feature maps for guiding segmentation. More specifically, we design an input decomposition module via a conditional variational auto-encoder to learn organ-specific…
Armed Forces Yangju Hospital（杨州武装部队医院）；Seoul National University（首尔国立大学）
For more clinical applications of deep learning models for medical image segmentation, high demands on labeled data and computational resources must be addressed. This study proposes a coarse-to-fine framework with two teacher models and a student model that combines knowledge distillation and cross teaching, a consistency regularization based on pseudo-labels, for efficient semi-supervised learning. The proposed method is demonstrated on the abdominal multi-organ segmentation task in CT images under the MICCAI FLARE 2022 challenge, with mean Dice scores of 0.8429 and 0.8520 in the validation …
Johns Hopkins University（约翰斯·霍普金斯大学）；Huazhong University of Science and Technology（华中科技大学）；Shanghai University（上海大学）；The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
Deep convolutional neural networks (CNNs), especially fully convolutional networks, have been widely applied to automatic medical image segmentation problems, e.g., multi-organ segmentation. Existing CNN-based segmentation methods mainly focus on looking for increasingly powerful network architectures, but pay less attention to data sampling strategies for training networks more effectively. In this paper, we present a simple but effective sample selection method for training multi-organ segmentation networks. Sample selection exhibits an exploitation-exploration strategy, i.e., exploiting har…
Osaka University Graduate School of Medicine（大阪大学医学研究生院）
Multi-organ segmentation enables organ evaluation, accounts the relationship between multiple organs, and facilitates accurate diagnosis and treatment decisions. However, only few models can perform segmentation accurately because of the lack of datasets and computational resources. On AMOS2022 challenge, which is a large-scale, clinical, and diverse abdominal multiorgan segmentation benchmark, we trained a 3D-UNet model with large batch and patch sizes using multi-GPU distributed training. Segmentation performance tended to increase for models with large batch and patch sizes compared with th…
University of Electronic Science and Technology of China（电子科技大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Sichuan Cancer Hospital and Institute（四川省肿瘤医院（四川省肿瘤研究所））；Sichuan Cancer Center（四川省癌症中心）；Sichuan University（四川大学）；West China Hospital（华西医院）；Johns Hopkins University（约翰斯·霍普金斯大学）；SenseTime Research（商汤科技研究院）；Rutgers University（罗格斯大学）
Whole abdominal organ segmentation is important in diagnosing abdomen lesions, radiotherapy, and follow-up. However, oncologists' delineating all abdominal organs from 3D volumes is time-consuming and very expensive. Deep learning-based medical image segmentation has shown the potential to reduce manual delineation efforts, but it still requires a large-scale fine annotated dataset for training, and there is a lack of large-scale datasets covering the whole abdomen region with accurate and detailed annotations for the whole abdominal organ segmentation. In this work, we establish a new large-s…
Fosun Aitrox Information Technology Co LTD（复星艾特罗克斯信息技术有限公司）
The contextual information, presented in abdominal CT scan, is relative consistent. In order to make full use of the overall 3D context, we develop a whole-volume-based coarse-to-fine framework for efficient and effective abdominal multi-organ segmentation. We propose a new efficientSegNet network, which is composed of basic encoder, slim decoder and efficient context block. For the decoder module, anisotropic convolution with a k*k*1 intra-slice convolution and a 1*1*k inter-slice convolution, is designed to reduce the computation burden. For the context block, we propose strip pooling module…
