Automatic Data Augmentation via Deep Reinforcement Learning for Effective Kidney Tumor Segmentation
Nanjing University（南京大学）；Medical School of Nanjing University（南京大学医学院）；National Institute of Healthcare Data Science, Nanjing University（南京大学国家健康数据科学研究院）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）
Conventional data augmentation realized by performing simple pre-processing operations (\eg, rotation, crop, \etc) has been validated for its advantage in enhancing the performance for medical image segmentation. However, the data generated by these conventional augmentation methods are random and sometimes harmful to the subsequent segmentation. In this paper, we developed a novel automatic learning-based data augmentation method for medical image segmentation which models the augmentation task as a trial-and-error procedure using deep reinforcement learning (DRL). In our method, we innovatively combine the data augmentation module and the subsequent segmentation module in an end-to-end training manner with a consistent loss. Specifically, the best sequential combination of different basic operations is automatically learned by directly maximizing the performance improvement (\ie, Dice ratio) on the available validation set. We extensively evaluated our method on CT kidney tumor segmentation which validated the promising results of our method.
NVIDIA（英伟达）；University of California, Los Angeles（加州大学洛杉矶分校）
Automated segmentation of kidneys and kidney tumors is an important step in quantifying the tumor's morphometrical details to monitor the progression of the disease and accurately compare decisions regarding the kidney tumor treatment. Manual delineation techniques are often tedious, error-prone and require expert knowledge for creating unambiguous representation of kidneys and kidney tumors segmentation. In this work, we propose an end-to-end boundary aware fully Convolutional Neural Networks (CNNs) for reliable kidney and kidney tumor semantic segmentation from arterial phase abdominal 3D CT…
Couger Inc（库格公司）；Sethu Institute of Technology（塞图理工学院）；Thiagarajar college of Engineering（蒂亚加拉贾工程学院）
KiTs19 challenge paves the way to haste the improvement of solid kidney tumor semantic segmentation methodologies. Accurate segmentation of kidney tumor in computer tomography (CT) images is a challenging task due to the non-uniform motion, similar appearance and various shape. Inspired by this fact, in this manuscript, we present a novel kidney tumor segmentation method using deep learning network termed as Hyper vision Net model. All the existing U-net models are using a modified version of U-net to segment the kidney tumor region. In the proposed architecture, we introduced supervision laye…
University of Minnesota（明尼苏达大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；University of Heidelberg（海德堡大学）；PingAn Technology Co., Ltd（平安科技有限公司）；Shanghai United Imaging Intelligence Inc.（上海联影智能医疗科技有限公司）；Southern Medical University（南方医科大学）；Peking University First Hospital（北京大学第一医院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
There is a large body of literature linking anatomic and geometric characteristics of kidney tumors to perioperative and oncologic outcomes. Semantic segmentation of these tumors and their host kidneys is a promising tool for quantitatively characterizing these lesions, but its adoption is limited due to the manual effort required to produce high-quality 3D segmentations of these structures. Recently, methods based on deep learning have shown excellent results in automatic 3D segmentation, but they require large datasets for training, and there remains little consensus on which methods perform…
U-Net has achieved huge success in various medical image segmentation challenges. Kinds of new architectures with bells and whistles might succeed in certain dataset when employed with optimal hyper-parameter, but their generalization always can't be guaranteed. Here, we focused on the basic U-Net architecture and proposed a multi scale supervised 3D U-Net for the segmentation task in KiTS19 challenge. To enhance the performance, our work can be summarized as three folds: first, we used multi scale supervision in the decoder pathway, which could encourage the network to predict right results f…
The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT
University of Minnesota（明尼苏达大学）；German Cancer Research Center（德国癌症研究中心）；Shanghai Jiao Tong University（上海交通大学）；IBM Research（IBM研究院）；Monash University（莫纳什大学）；Northwestern Polytechnical University（西北工业大学）；The University of Sydney（悉尼大学）；Nankai University（南开大学）
College of Biomedical Engineering, Rangsit University（兰实大学生物医学工程学院）
Automated medical image segmentation is a priority research area for computational methods. In particular, detection of cancerous tumors represents a current challenge in this area with potential for real-world impact. This paper describes a method developed in response to the 2019 Kidney Tumor Segmentation Challenge (KiTS19). Axial computed tomography (CT) scans from 210 kidney cancer patients were used to develop and evaluate this automatic segmentation method based on a logical ensemble of fully-convolutional network (FCN) architectures, followed by volumetric validation. Data was pre-proce…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Southeast University（东南大学）；Lenovo Research（联想研究院）
Automated segmentation of kidney and tumor from 3D CT scans is necessary for the diagnosis, monitoring, and treatment planning of the disease. In this paper, we describe a two-stage framework for kidney and tumor segmentation based on 3D fully convolutional network (FCN). The first stage preliminarily locate the kidney and cut off the irrelevant background to reduce class imbalance and computation cost. Then the second stage precisely segment the kidney and tumor on the cropped patch. The proposed method ranks the 4th place out of 105 competitive teams in MICCAI 2019 KiTS Challenge with a Comp…
Image Sciences Institute, UMC Utrecht, Utrecht University（乌得勒支大学医学中心影像科学研究所）
This paper assesses whether using clinical characteristics in addition to imaging can improve automated segmentation of kidney cancer on contrast-enhanced computed tomography (CT). A total of 300 kidney cancer patients with contrast-enhanced CT scans and clinical characteristics were included. A baseline segmentation of the kidney cancer was performed using a 3D U-Net. Input to the U-Net were the contrast-enhanced CT images, output were segmentations of kidney, kidney tumors, and kidney cysts. A cognizant sampling strategy was used to leverage clinical characteristics for improved segmentation…
ARTORG Center for Biomedical Engineering Research, University of Bern（伯尔尼大学生物医学工程研究中心）
Each year, there are about 400'000 new cases of kidney cancer worldwide causing around 175'000 deaths. For clinical decision making it is important to understand the morphometry of the tumor, which involves the time-consuming task of delineating tumor and kidney in 3D CT images. Automatic segmentation could be an important tool for clinicians and researchers to also study the correlations between tumor morphometry and clinical outcomes. We present a segmentation method which combines the popular U-Net convolutional neural network architecture with post-processing based on statistical constrain…
Precise characterization of the kidney and kidney tumor characteristics is of outmost importance in the context of kidney cancer treatment, especially for nephron sparing surgery which requires a precise localization of the tissues to be removed. The need for accurate and automatic delineation tools is at the origin of the KiTS19 challenge. It aims at accelerating the research and development in this field to aid prognosis and treatment planning by providing a characterized dataset of 300 CT scans to be segmented. To address the challenge, we proposed an automatic, multi-stage, 2.5D deep learn…
James Cook University（詹姆斯·库克大学）；Mahidol University（玛希隆大学）；University of Bremen（不来梅大学）
Automatic segmentation of hepatic lesions in computed tomography (CT) images is a challenging task to perform due to heterogeneous, diffusive shape of tumors and complex background. To address the problem more and more researchers rely on assistance of deep convolutional neural networks (CNN) with 2D or 3D type architecture that have proven to be effective in a wide range of computer vision tasks, including medical image processing. In this technical report, we carry out research focused on more careful approach to the process of learning rather than on complex architecture of the CNN. We have…
Kidney structures segmentation is a crucial yet challenging task in the computer-aided diagnosis of surgery-based renal cancer. Although numerous deep learning models have achieved remarkable success in many medical image segmentation tasks, accurate segmentation of kidney structures on computed tomography angiography (CTA) images remains challenging, due to the variable sizes of kidney tumors and the ambiguous boundaries between kidney structures and their surroundings. In this paper, we propose a boundary-aware network (BA-Net) to segment kidneys, kidney tumors, arteries, and veins on CTA sc…
Umeå University（于默奥大学）；Eindhoven University of Technology（埃因霍温理工大学）
Kidney tumor segmentation emerges as a new frontier of computer vision in medical imaging. This is partly due to its challenging manual annotation and great medical impact. Within the scope of the Kidney Tumor Segmentation Challenge 2019, that is aiming at combined kidney and tumor segmentation, this work proposes a novel combination of 3D U-Nets---collectively denoted TuNet---utilizing the resulting kidney masks for the consecutive tumor segmentation. The proposed method achieves a Sørensen-Dice coefficient score of 0.902 for the kidney, and 0.408 for the tumor segmentation, computed from a f…
University of Oxford（牛津大学）；Oxford University Hospitals NHS Trust（牛津大学医院NHS信托基金会）
Kidney stones represent a considerable burden for public health-care systems. Ureteroscopy with laser lithotripsy has evolved as the most commonly used technique for the treatment of kidney stones. Automated segmentation of kidney stones and laser fiber is an important initial step to performing any automated quantitative analysis of the stones, particularly stone-size estimation, that helps the surgeon decide if the stone requires more fragmentation. Factors such as turbid fluid inside the cavity, specularities, motion blur due to kidney movements and camera motion, bleeding, and stone debris…
Technical University of Kosice（科希策工业大学）；Pavol Jozef Safarik University（帕沃尔·约瑟夫·沙法里克大学）；Louis Pasteur University Hospital（路易斯·巴斯德大学医院）
The kidney cancer is one of the most common cancer types. The treatment frequently include surgical intervention. However, surgery is in this case particularly challenging due to regional anatomical relations. Organ delineation can significantly improve surgical planning and execution. In this contribution, we propose ensemble of two fully convolutional networks for segmentation of kidney, tumor, veins and arteries. While SegResNet architecture achieved better performance on tumor, the nnU-Net provided more precise segmentation for kidneys, arteries and veins. So in our proposed approach we co…
Accurate segmentation of kidneys and kidney tumors is an essential step for radiomic analysis as well as developing advanced surgical planning techniques. In clinical analysis, the segmentation is currently performed by clinicians from the visual inspection images gathered through a computed tomography (CT) scan. This process is laborious and its success significantly depends on previous experience. Moreover, the uncertainty in the tumor location and heterogeneity of scans across patients increases the error rate. To tackle this issue, computer-aided segmentation based on deep learning techniq…
Peking University（北京大学）；Canon Medical Systems（佳能医疗系统公司）
Data augmentation is an effective and universal technique for improving generalization performance of deep neural networks. It could enrich diversity of training samples that is essential in medical image segmentation tasks because 1) the scale of medical image dataset is typically smaller, which may increase the risk of overfitting; 2) the shape and modality of different objects such as organs or tumors are unique, thus requiring customized data augmentation policy. However, most data augmentation implementations are hand-crafted and suboptimal in medical image processing. To fully exploit th…
UNesT: Local Spatial Representation Learning with Hierarchical Transformer for Efficient Medical Segmentation
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；Siemens Healthineers（西门子医疗）；Annalise-AI, Pty, Ltd（Annalise人工智能私人有限公司）；Google Cloud AI（谷歌云人工智能）
Robust Renal Mass Segmentation on CT: A Validation Study of an AI-Based Framework
Department of Medical Imaging, Radboudumc, Nijmegen, The Netherlands（医学影像部门，Radboudumc，尼姆维根，荷兰）；Department of Radiology, Charité - Universitätsmedizin Berlin, Berlin, Germany（放射科，Charité - 大学医学中心柏林，柏林，德国）；Department of Neuroradiology, Charité - Universitätsmedizin Berlin, Berlin, Germany（神经放射科，Charité - 大学医学中心柏林，柏林，德国）；Department of Diagnostic and Interventional Radiology, Klinikum rechts der Isar, TUM University Hospital, Technical University of Munich, Munich, Germany（诊断和介入放射科，Klinikum rechts der Isar，TUM大学医院，慕尼黑技术大学，慕尼黑，德国）；Department of Cardiovascular Radiology and Nuclear Medicine, German Heart Center, TUM University Hospital, Technical University of Munich, Munich, Germany（心血管放射学和核医学部，德国心脏中心，TUM大学医院，慕尼黑技术大学，慕尼黑，德国）；Fraunhofer MEVIS, Bremen, Germany（Fraunhofer MEVIS，不莱梅，德国）
Hybrid Deep Learning Framework for Classification of Kidney CT Images: Diagnosis of Stones, Cysts, and Tumors
School of Engineering & Technology, BML Munjal University（BML Munjal大学工程与技术学院）；Center for Advanced Data and Computational Science, BML Munjal University（BML Munjal大学先进数据与计算科学中心）
Three-dimensional (3D) kidney parsing on computed tomography angiography (CTA) images is of great clinical significance. Automatic segmentation of kidney, renal tumor, renal vein and renal artery benefits a lot on surgery-based renal cancer treatment. In this paper, we propose a new nnhra-unet network, and use a multi-stage framework which is based on it to segment the multi-structure of kidney and participate in the KiPA2022 challenge.
R$^{2}$Seg: Training-Free OOD Medical Tumor Segmentation via Anatomical Reasoning and Statistical Rejection
Carnegie Mellon University（卡内基梅隆大学）；University of Cambridge（剑桥大学）；Zhejiang University（浙江大学）；ETH Zurich（苏黎世联邦理工学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Automatic brain tumor segmentation from Magnetic Resonance Imaging (MRI) data plays an important role in assessing tumor response to therapy and personalized treatment stratification.Manual segmentation is tedious and subjective.Deep-learning-based algorithms for brain tumor segmentation have the potential to provide objective and fast tumor segmentation.However, the training of such algorithms requires large datasets which are not always available. Data augmentation techniques may reduce the need for large datasets.However current approaches are mostly parametric and may result in suboptimal …
Center for Medical Image Science and Visualization (CMIV)（医学图像科学与可视化中心）
Training segmentation networks requires large annotated datasets, which in medical imaging can be hard to obtain. Despite this fact, data augmentation has in our opinion not been fully explored for brain tumor segmentation. In this project we apply different types of data augmentation (flipping, rotation, scaling, brightness adjustment, elastic deformation) when training a standard 3D U-Net, and demonstrate that augmentation significantly improves the network's performance in many cases. Our conclusion is that brightness augmentation and elastic deformation work best, and that combinations of …
Korea Advanced Institute of Science and Technology (KAIST)（韩国高等科学技术研究院（KAIST））
Medical image analysis using deep neural networks has been actively studied. Deep neural networks are trained by learning data. For accurate training of deep neural networks, the learning data should be sufficient, of good quality, and should have a generalized property. However, in medical images, it is difficult to acquire sufficient patient data because of the difficulty of patient recruitment, the burden of annotation of lesions by experts, and the invasion of patients' privacy. In comparison, the medical images of healthy volunteers can be easily acquired. Using healthy brain images, the …
MSWAL: 3D Multi-class Segmentation of Whole Abdominal Lesions Dataset
Shanghai AI Lab（上海人工智能实验室）；University of Liverpool（利物浦大学）；Columbia university（哥伦比亚大学）；Monash University（墨尔本大学）；Xi'an Jiaotong-Liverpool University（西安交通大学利物浦大学）；MBZUAI（穆斯林人工智能研究所）；People's Hospital of Gong'an County（公安县人民医院）
本文提出首个 3D 全腹部病变多类分割数据集 MSWAL，覆盖七类常见病变，并提出 Inception nnU-Net 框架，在分割性能上显著优于现有方法。
Tianjin University（天津大学）；CSIRO Data61（澳大利亚联邦科学与工业研究组织Data61）；La Trobe University（乐卓博大学）；Tianjin University of Traditional Chinese Medicine（天津中医药大学）
The insufficiency of annotated medical imaging scans for cancer makes it challenging to train and validate data-hungry deep learning models in precision oncology. We propose a new richer generative adversarial network for free-form 3D tumor/lesion synthesis in computed tomography (CT) images. The network is composed of a new richer convolutional feature enhanced dilated-gated generator (RicherDG) and a hybrid loss function. The RicherDG has dilated-gated convolution layers to enable tumor-painting and to enlarge perceptive fields; and it has a novel richer convolutional feature association bra…
Hebei University of Technology（河北工业大学）；University of Macau（澳门大学）；University of Oxford（牛津大学）；Harbin Institute of Technology（哈尔滨工业大学）
Recently deep learning methods, in particular, convolutional neural networks (CNNs), have led to a massive breakthrough in the range of computer vision. Also, the large-scale annotated dataset is the essential key to a successful training procedure. However, it is a huge challenge to get such datasets in the medical domain. Towards this, we present a data augmentation method for generating synthetic medical images using cycle-consistency Generative Adversarial Networks (GANs). We add semi-supervised attention modules to generate images with convincing details. We treat tumor images and normal …
Huazhong University of Science and Technology（华中科技大学）；Kennesaw State University（肯尼索州立大学）
The crossMoDA challenge aims to automatically segment the vestibular schwannoma (VS) tumor and cochlea regions of unlabeled high-resolution T2 scans by leveraging labeled contrast-enhanced T1 scans. The 2022 edition extends the segmentation task by including multi-institutional scans. In this work, we proposed an unpaired cross-modality segmentation framework using data augmentation and hybrid convolutional networks. Considering heterogeneous distributions and various image sizes for multi-institutional scans, we apply the min-max normalization for scaling the intensities of all scans between …
Beijing Technology and Business University（北京工商大学）
Automatic segmentation of glioma and its subregions is of great significance for diagnosis, treatment and monitoring of disease. In this paper, an augmentation method, called TensorMixup, was proposed and applied to the three dimensional U-Net architecture for brain tumor segmentation. The main ideas included that first, two image patches with size of 128 in three dimensions were selected according to glioma information of ground truth labels from the magnetic resonance imaging data of any two patients with the same modality. Next, a tensor in which all elements were independently sampled from…
Department of Science and Technology, Linkoping University（林雪平大学科技学院）；Sectra AB（Sectra公司）；Center for Medical Image Science and Visualization, Linkoping University（林雪平大学医学影像科学与可视化中心）
The scarcity of labeled data is a major bottleneck for developing accurate and robust deep learning-based models for histopathology applications. The problem is notably prominent for the task of metastasis detection in lymph nodes, due to the tissue's low tumor-to-non-tumor ratio, resulting in labor- and time-intensive annotation processes for the pathologists. This work explores alternatives on how to augment the training data for colon carcinoma metastasis detection when there is limited or no representation of the target domain. Through an exhaustive study of cross-validated experiments wit…
University of Chinese Academy of Sciences（中国科学院大学）；Emory University（埃默里大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Southern California（南加利福尼亚大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
How We Won BraTS-SSA 2025: Brain Tumor Segmentation in the Sub-Saharan African Population Using Segmentation-Aware Data Augmentation and Model Ensembling
Kwame Nkrumah University of Science and Technology（卡布雷·克鲁马大学科学与技术大学）；University of Cape Coast（科克斯特大学）；University of Toronto（多伦多大学）；Department of Biomedical Engineering, McGill University（麦吉尔大学生物医学工程系）；Department of Physics, Federal University of Technology（联邦技术大学物理系）
University of Houston（休斯顿大学）；University Hospital of Cologne（科隆大学医院）
The classification of Antibody Mediated Rejection (AMR) in kidney transplant remains challenging even for experienced nephropathologists; this is partly because histological tissue stain analysis is often characterized by low inter-observer agreement and poor reproducibility. One of the implicated causes for inter-observer disagreement is the variability of tissue stain quality between (and within) pathology labs, coupled with the gradual fading of archival sections. Variations in stain colors and intensities can make tissue evaluation difficult for pathologists, ultimately affecting their abi…
Numerous oncology indications have extensively quantified metabolically active tumors using positron emission tomography (PET) and computed tomography (CT). F-fluorodeoxyglucose-positron emission tomography (FDG-PET) is frequently utilized in clinical practice and clinical drug research to detect and measure metabolically active malignancies. The assessment of tumor burden using manual or computer-assisted tumor segmentation in FDG-PET images is widespread. Deep learning algorithms have also produced effective solutions in this area. However, there may be a need to improve the performance of a…
A Multi-Stage Deep Learning Framework with PKCP-MixUp Augmentation for Pediatric Liver Tumor Diagnosis Using Multi-Phase Contrast-Enhanced CT
Wuhan Children's Hospital (Wuhan Maternal and Child Healthcare Hospital), Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院武汉儿童医院（武汉市妇幼保健院））；Wuhan University（武汉大学）；Wuhan Cloud Computing Technology Co., Ltd.（武汉云计算技术有限公司）
Tumor-aware augmentation with task-guided attention analysis improves rectal cancer segmentation from magnetic resonance images
Department of Medical Physics, Memorial Sloan Kettering Cancer Center（医学物理系，纪念斯隆凯特琳癌症中心）；Department of Radiology, Memorial Sloan Kettering Cancer Center（放射学系，纪念斯隆凯特琳癌症中心）；Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center（放射肿瘤学系，纪念斯隆凯特琳癌症中心）；Department of Surgery, Memorial Sloan Kettering Cancer Center（外科系，纪念斯隆凯特琳癌症中心）；Department of Medicine, Memorial Sloan Kettering Cancer Center（医学系，纪念斯隆凯特琳癌症中心）
German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；University of Heidelberg（海德堡大学）
The U-Net is arguably the most successful segmentation architecture in the medical domain. Here we apply a 3D U-Net to the 2019 Kidney and Kidney Tumor Segmentation Challenge and attempt to improve upon it by augmenting it with residual and pre-activation residual blocks. Cross-validation results on the training cases suggest only very minor, barely measurable improvements. Due to marginally higher dice scores, the residual 3D U-Net is chosen for test set prediction. With a Composite Dice score of 91.23 on the test set, our method outperformed all 105 competing teams and won the KiTS2019 chall…
Johns Hopkins University（约翰霍普金斯大学）；University of Bologna（博洛尼亚大学）；Italian Institute of Technology（意大利理工学院）；University of California, San Francisco（加州大学旧金山分校）；Istanbul Medipol University（伊斯坦布尔Medipol大学）；University of Zurich（苏黎世大学）；ETH AI Center（ETH人工智能中心）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Heidelberg University Hospital（海德堡大学医院）
We apply nnU-Net to the segmentation task of the BraTS 2020 challenge. The unmodified nnU-Net baseline configuration already achieves a respectable result. By incorporating BraTS-specific modifications regarding postprocessing, region-based training, a more aggressive data augmentation as well as several minor modifications to the nnUNet pipeline we are able to improve its segmentation performance substantially. We furthermore re-implement the BraTS ranking scheme to determine which of our nnU-Net variants best fits the requirements imposed by it. Our final ensemble took the first place in the…
