Due to the intensive cost of labor and expertise in annotating 3D medical images at a voxel level, most benchmark datasets are equipped with the annotations of only one type of organs and/or tumors, resulting in the so-called partially labeling issue. To address this, we propose a dynamic on-demand network (DoDNet) that learns to segment multiple organs and tumors on partially labeled datasets. DoDNet consists of a shared encoder-decoder architecture, a task encoding module, a controller for generating dynamic convolution filters, and a single but dynamic segmentation head. The information of the current segmentation task is encoded as a task-aware prior to tell the model what the task is expected to solve. Different from existing approaches which fix kernels after training, the kernels in dynamic head are generated adaptively by the controller, conditioned on both input image and assigned task. Thus, DoDNet is able to segment multiple organs and tumors, as done by multiple networks or a multi-head network, in a much efficient and flexible manner. We have created a large-scale partially labeled dataset, termed MOTS, and demonstrated the superior performance of our DoDNet over other competitors on seven organ and tumor segmentation tasks. We also transferred the weights pre-trained on MOTS to a downstream multi-organ segmentation task and achieved state-of-the-art performance. This study provides a general 3D medical image segmentation model that has been pre-trained on a large-scale partially labelled dataset and can be extended (after fine-tuning) to downstream volumetric medical data segmentation tasks. The dataset and code areavailableat: https://git.io/DoDNet
Northwestern Polytechnical University（西北工业大学）；The University of Adelaide（阿德莱德大学）；Alibaba DAMO Academy（阿里巴巴达摩院）；Zhejiang University（浙江大学）
Medical image benchmarks for the segmentation of organs and tumors suffer from the partially labeling issue due to its intensive cost of labor and expertise. Current mainstream approaches follow the practice of one network solving one task. With this pipeline, not only the performance is limited by the typically small dataset of a single task, but also the computation cost linearly increases with the number of tasks. To address this, we propose a Transformer based dynamic on-demand network (TransDoDNet) that learns to segment organs and tumors on multiple partially labeled datasets. Specifical…
Southeast University（东南大学）；Swinburne University of Technology（斯威本科技大学）；Communication University of China（中国传媒大学）
Precise segmentation of organs and tumors plays a crucial role in clinical applications. It is a challenging task due to the irregular shapes and various sizes of organs and tumors as well as the significant class imbalance between the anatomy of interest (AOI) and the background region. In addition, in most situation tumors and normal organs often overlap in medical images, but current approaches fail to delineate both tumors and organs accurately. To tackle such challenges, we propose a position-guided deformable UNet, namely PGD-UNet, which exploits the spatial deformation capabilities of d…
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Pancreas Centre, First Affiliated Hospital, Nanjing Medical University（南京医科大学第一附属医院胰腺中心）；Department of General Surgery, First Affiliated Hospital, Nanjing Medical University（南京医科大学第一附属医院普外科）；Big Data Computing Center of Southeast University（东南大学大数据计算中心）
Unleashing Diffusion and State Space Models for Medical Image Segmentation
Department of Biostatistics, School of Global Public Health, New York University（生物统计学系、全球公共卫生学院、纽约大学）；School of Statistics, KLATASDS-MOE, East China Normal University（统计学系、KLATASDS-MOE、华东师范大学）；School of Biomedical Engineering, Southern Medical University（生物医学工程学院、南方医科大学）；Faculty of Biomedical Engineering, Shenzhen University of Advanced Technology（生物医学工程学院、深圳先进技术大学）
With the development of image segmentation in computer vision, biomedical image segmentation have achieved remarkable progress on brain tumor segmentation and Organ At Risk (OAR) segmentation. However, most of the research only uses single modality such as Computed Tomography (CT) scans while in real world scenario doctors often use multiple modalities to get more accurate result. To better leverage different modalities, we have collected a large dataset consists of 136 cases with CT and MR images which diagnosed with nasopharyngeal cancer. In this paper, we propose to use Generative Adversari…
NVIDIA（英伟达）；University of California, Los Angeles（加州大学洛杉矶分校）
Automated segmentation of kidneys and kidney tumors is an important step in quantifying the tumor's morphometrical details to monitor the progression of the disease and accurately compare decisions regarding the kidney tumor treatment. Manual delineation techniques are often tedious, error-prone and require expert knowledge for creating unambiguous representation of kidneys and kidney tumors segmentation. In this work, we propose an end-to-end boundary aware fully Convolutional Neural Networks (CNNs) for reliable kidney and kidney tumor semantic segmentation from arterial phase abdominal 3D CT…
HBFormer: A Hybrid-Bridge Transformer for Microtumor and Miniature Organ Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；Guangdong University of Technology（广东技术大学）；University of Macau（澳门大学）；Huizhou University（惠州大学）
University College London（伦敦大学学院）；KU Leuven（鲁汶大学）；University Hospitals KU Leuven（鲁汶大学医院）
Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes. To address these problems, we propose a novel deep learning-based framework for interactive segmentation by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine-tuning to make a CNN m…
R$^{2}$Seg: Training-Free OOD Medical Tumor Segmentation via Anatomical Reasoning and Statistical Rejection
Carnegie Mellon University（卡内基梅隆大学）；University of Cambridge（剑桥大学）；Zhejiang University（浙江大学）；ETH Zurich（苏黎世联邦理工学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Medical imaging is the most important tool for detecting complications in the inner body of medicine. Nowadays, with the development of image processing technology as well as changing the size of photos to higher resolution images in the field of digital medical imaging, there is an efficient and accurate system for segmenting this. Real-world images that for a variety of reasons have poor heterogeneity, noise and contrast are essential. Digital image segmentation in medicine is used for diagnostic and therapeutic analysis, which is very helpful for physicians. In this study, we aim at liver c…
Memorial Sloan Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）
Purpose: Radiotherapy presents unique challenges and clinical requirements for longitudinal tumor and organ-at-risk (OAR) prediction during treatment. The challenges include tumor inflammation/edema and radiation-induced changes in organ geometry, whereas the clinical requirements demand flexibility in input/output sequence timepoints to update the predictions on rolling basis and the grounding of all predictions in relationship to the pre-treatment imaging information for response and toxicity assessment in adaptive radiotherapy. Methods: To deal with the aforementioned challenges and to comp…
Self-Supervised Learning for Organs At Risk and Tumor Segmentation with Uncertainty Quantification
University of Central Florida（中佛罗里达大学）；Northwestern University（西北大学）；KnowledgeVis LLC（KnowledgeVis有限责任公司）；Orlando Health（奥兰多健康）；University of Chicago（芝加哥大学）
Technical University of Denmark（丹麦技术大学）；Skåne University Hospital（斯科讷大学医院）；Danish Research Centre for Magnetic Resonance（丹麦磁共振研究中心）；Copenhagen University Hospital Hvidovre（哥本哈根大学医院维兹奥勒医院）；Copenhagen University Hospital Rigshospitalet（哥本哈根大学医院国立医院）；University College London（伦敦大学学院）；UCLH NHS Foundation Trust（伦敦大学学院医院NHS基金会信托）；National Hospital for Neurology and Neurosurgery（国立神经病学与神经外科医院）
In this paper we present a method for simultaneously segmenting brain tumors and an extensive set of organs-at-risk for radiation therapy planning of glioblastomas. The method combines a contrast-adaptive generative model for whole-brain segmentation with a new spatial regularization model of tumor shape using convolutional restricted Boltzmann machines. We demonstrate experimentally that the method is able to adapt to image acquisitions that differ substantially from any available training data, ensuring its applicability across treatment sites; that its tumor segmentation accuracy is compara…
Merlin Plus: A Large-Scale, Multi-Cancer, Image-Mask-Report Dataset
Johns Hopkins University（约翰斯·霍普金斯大学）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（马萨诸塞总医院）；Jagiellonian University（雅盖隆大学）；Stanford University（斯坦福大学）；Warmian-Masurian Cancer Center（瓦尔米亚-马祖里癌症中心）；University of California, San Francisco（加利福尼亚大学旧金山分校）；Johns Hopkins Medicine（约翰斯·霍普金斯医学）
Northwestern University（西北大学）；University of Chicago（芝加哥大学）
Head and Neck (H\&N) organ-at-risk (OAR) and tumor segmentations are essential components of radiation therapy planning. The varying anatomic locations and dimensions of H\&N nodal Gross Tumor Volumes (GTVn) and H\&N primary gross tumor volume (GTVp) are difficult to obtain due to lack of accurate and reliable delineation methods. The downstream effect of incorrect segmentation can result in unnecessary irradiation of normal organs. Towards a fully automated radiation therapy planning algorithm, we explore the efficacy of multi-scale fusion based deep learning architectures for accurately segm…
Department of Science and Technology, Linkoping University（林雪平大学科技学院）；Sectra AB（Sectra公司）；Center for Medical Image Science and Visualization, Linkoping University（林雪平大学医学影像科学与可视化中心）
The scarcity of labeled data is a major bottleneck for developing accurate and robust deep learning-based models for histopathology applications. The problem is notably prominent for the task of metastasis detection in lymph nodes, due to the tissue's low tumor-to-non-tumor ratio, resulting in labor- and time-intensive annotation processes for the pathologists. This work explores alternatives on how to augment the training data for colon carcinoma metastasis detection when there is limited or no representation of the target domain. Through an exhaustive study of cross-validated experiments wit…
Yokohama City University Medical Center（横滨市立大学医学中心）
We present a fully automatic method employing convolutional neural networks based on the 2D U-net architecture and random forest classifier to solve the automatic liver lesion segmentation problem of the ISBI 2017 Liver Tumor Segmentation Challenge (LiTS). In order to constrain the ROI in which the tumors could be located, a liver segmentation is performed first. For the organ segmentation, an ensemble of convolutional networks is trained to segment a liver using a set of 179 liver CT datasets from liver surgery planning. Inside of the liver ROI a neural network, trained using 127 challenge tr…
Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院北京协和医院）；Hupan Laboratory（湖畔实验室）；Fuzhou University（福州大学）；Eye & ENT Hospital, Fudan University（复旦大学附属眼耳鼻喉科医院）；Zhongshan Hospital, Fudan University（复旦大学附属中山医院）；Chang Gung Memorial Hospital（长庚纪念医院）；Zhongshan Hospital(Xiamen), Fudan University（复旦大学附属中山医院厦门医院）
CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
City University of Hong Kong（香港城市大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Vanderbilt University（范德堡大学）；Chinese University of Hong Kong（香港中文大学）；CUHK Shenzhen Research Institute（香港中文大学深圳研究院）；NVIDIA（英伟达公司）
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（慕尼黑 Rechts der Isar 医院）；University of Zurich（苏黎世大学）
Automatic localization and segmentation of organs-at-risk (OAR) in CT are essential pre-processing steps in medical image analysis tasks, such as radiation therapy planning. For instance, the segmentation of OAR surrounding tumors enables the maximization of radiation to the tumor area without compromising the healthy tissues. However, the current medical workflow requires manual delineation of OAR, which is prone to errors and is annotator-dependent. In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation rather than novel localization or segmentation archite…
CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Johns Hopkins University（约翰斯·霍普金斯大学）；Hupan Lab（湖畔实验室）；Sun Yat-sen University Cancer Center（中山大学肿瘤防治中心）；Sichuan Cancer Hospital（四川省肿瘤医院）；The First Affiliated Hospital of Zhejiang University（浙江大学医学院附属第一医院）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；Guangdong Provincial People’s Hospital（广东省人民医院）
Enhancing Renal Tumor Malignancy Prediction: Deep Learning with Automatic 3D CT Organ Focused Attention
1 Department of Health Outcomes \& Biomedical Informatics, University of Florida, Gainesville, FL, USA 2 Department of Urology, University of Florida, Gainesville, FL, USA 3 Department of Computer \& Information Sciences, Temple University, Philadelphia, PA, USA
Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data
Johns Hopkins University（约翰霍普金斯大学）；UCAS-Terminus AI Lab, University of Chinese Academy of Sciences（中国科学院大学Terminus AI实验室）；Hong Kong Polytechnic University（香港理工大学）；University of Cambridge（剑桥大学）；Sichuan University（四川大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；Qilu Hospital of Shandong University（山东大学齐鲁医院）；Westlake University（西湖大学）
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea（延世大学电气与计算机工程系）；Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, Suwon, Korea（延世大学医学院放射科与成像科学中心）
Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography
City University of Hong Kong（香港城市大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of California, San Francisco（加州大学旧金山分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Chinese University of Hong Kong（香港中文大学）；NVIDIA（英伟达）
PAII Inc.（PAII公司）；Chang Gung Memorial Hospital（长庚纪念医院）；Ping An Technology（平安科技）
Clinical target volume (CTV) delineation from radiotherapy computed tomography (RTCT) images is used to define the treatment areas containing the gross tumor volume (GTV) and/or sub-clinical malignant disease for radiotherapy (RT). High intra- and inter-user variability makes this a particularly difficult task for esophageal cancer. This motivates automated solutions, which is the aim of our work. Because CTV delineation is highly context-dependent--it must encompass the GTV and regional lymph nodes (LNs) while also avoiding excessive exposure to the organs at risk (OARs)--we formulate it as a…
School of Biomedical Engineering（生物医学工程学院）；Universidad Autónoma de Ciudad Juárez（华雷斯自治大学）
The brain is a complex organ controlling cognitive process and physical functions. Tumors in the brain are accelerated cell growths affecting the normal function and processes in the brain. MRI scans provides detailed images of the body being one of the most common tests to diagnose brain tumors. The process of segmentation of brain tumors from magnetic resonance imaging can provide a valuable guide for diagnosis, treatment planning and prediction of results. Here we consider the problem brain tumor segmentation using a Deep learning architecture for use in tumor segmentation. Although the pro…
Xiamen University（厦门大学）；The Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；The Hong Kong Polytechnic University（香港理工大学）；Manteia Technologies Co.,Ltd（Manteia科技有限公司）
Automatic delineation of organ-at-risk (OAR) and gross-tumor-volume (GTV) is of great significance for radiotherapy planning. However, it is a challenging task to learn powerful representations for accurate delineation under limited pixel (voxel)-wise annotations. Contrastive learning at pixel-level can alleviate the dependency on annotations by learning dense representations from unlabeled data. Recent studies in this direction design various contrastive losses on the feature maps, to yield discriminative features for each pixel in the map. However, pixels in the same map inevitably share sem…
National Institutes of Health（美国国立卫生研究院）；National Institute of Diabetes and Digestive and Kidney Diseases（国家糖尿病、消化和肾脏疾病研究所）
Finding small lesions is very challenging due to lack of noticeable features, severe class imbalance, as well as the size itself. One approach to improve small lesion segmentation is to reduce the region of interest and inspect it at a higher sensitivity rather than performing it for the entire region. It is usually implemented as sequential or joint segmentation of organ and lesion, which requires additional supervision on organ segmentation. Instead, we propose to utilize an intensity distribution of a target lesion at no additional labeling cost to effectively separate regions where the les…
Automatic segmentation of tumor lesions is a critical initial processing step for quantitative PET/CT analysis. However, numerous tumor lesion with different shapes, sizes, and uptake intensity may be distributed in different anatomical contexts throughout the body, and there is also significant uptake in healthy organs. Therefore, building a systemic PET/CT tumor lesion segmentation model is a challenging task. In this paper, we propose a novel training strategy to build deep learning models capable of systemic tumor segmentation. Our method is validated on the training set of the AutoPET 202…
Mansoura University（曼苏尔大学）；South Valley University（南河谷大学）；Higher Institute of Engineering and Technology（高等工程技术学院）
The liver is one of the most critical metabolic organs in vertebrates due to its vital functions in the human body, such as detoxification of the blood from waste products and medications. Liver diseases due to liver tumors are one of the most common mortality reasons around the globe. Hence, detecting liver tumors in the early stages of tumor development is highly required as a critical part of medical treatment. Many imaging modalities can be used as aiding tools to detect liver tumors. Computed tomography (CT) is the most used imaging modality for soft tissue organs such as the liver. This …
The most common primary brain tumors are gliomas, evolving from the cerebral supportive cells. For clinical follow-up, the evaluation of the preoperative tumor volume is essential. Volumetric assessment of tumor volume with manual segmentation of its outlines is a time-consuming process that can be overcome with the help of computerized segmentation methods. In this contribution, two methods for World Health Organization (WHO) grade IV glioma segmentation in the human brain are compared using magnetic resonance imaging (MRI) patient data from the clinical routine. One method uses balloon infla…
Stanford University（斯坦福大学）；Varian Medical Systems（瓦里安医疗系统公司）；Huiyihuiying Medical Co（慧医慧影医疗公司）；Google Inc（谷歌公司）
Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra-/inter- operator variations. While various algorithms for delineating organ-at-risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U-Net has gained increasing popularity recently for ima…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Brain tumor segmentation is essential for the diagnosis and prognosis of patients with gliomas. The brain tumor segmentation challenge has continued to provide a great source of data to develop automatic algorithms to perform the task. This paper describes our contribution to the 2021 competition. We developed our methods based on nn-UNet, the winning entry of last year competition. We experimented with several modifications, including using a larger network, replacing batch normalization with group normalization, and utilizing axial attention in the decoder. Internal 5-fold cross validation a…
SegRap2023: A Benchmark of Organs-at-Risk and Gross Tumor Volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma
University of Electronic Science and Technology of China（电子科技大学）；Sichuan Cancer Hospital & Institute（四川省肿瘤医院（研究所））；Shanghai AI Lab（上海人工智能实验室）；Canon Medical Systems (China) Co. Ltd.（佳能医疗系统（中国）有限公司）；Stockholm University（斯德哥尔摩大学）；KTH（瑞典皇家理工学院）；Northwestern Polytechnical University（西北工业大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）；Ping An Insurance Company of China（中国平安保险公司）
Developing an effective liver and liver tumor segmentation model from CT scans is very important for the success of liver cancer diagnosis, surgical planning and cancer treatment. In this work, we propose a two-stage framework for 2D liver and tumor segmentation. The first stage is a coarse liver segmentation network, while the second stage is an edge enhanced network (E$^2$Net) for more accurate liver and tumor segmentation. E$^2$Net explicitly models complementary objects (liver and tumor) and their edge information within the network to preserve the organ and lesion boundaries. We introduce…
T-MPEDNet: Unveiling the Synergy of Transformer-aware Multiscale Progressive Encoder-Decoder Network with Feature Recalibration for Tumor and Liver Segmentation
Department of Computer Science and Engineering, Indian Institute of Technology (IIT) Indore, Indore 453552, India（计算机科学与工程系，印度理工学院（IIT）印尔德，印尔德453552，印度）；Department of Electrical Engineering, Indian Institute of Technology (IIT) Indore, Indore 453552, India（电气工程系，印度理工学院（IIT）印尔德，印尔德453552，印度）
Iterative pseudo-labeling based adaptive copy-paste supervision for semi-supervised tumor segmentation
School of Software, Northwestern Polytechnical University, Shaanxi, China（西北工业大学软件学院）；Yangtze River Delta Research Institute of Northwestern Polytechnical University, Taicang, China（西北工业大学 Yangtze River Delta 研究院）；Department of Computer Science and Information Technology, La Trobe University, Melbourne, Australia（拉特罗布大学计算机科学与信息技术系）；Department of Computer Science, School of Engineering, Shantou University, Guangdong, China（汕尾大学工程学院计算机科学系）；Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Shandong, China（山东省肿瘤医院）；Australian Institute of Health Innovation (AIHI), Macquarie University, Australia（澳大利亚健康创新研究所（AIHI），麦考瑞大学）；School of Software, Shandong University, Shandong, China（山东大学软件学院）；AIDD, Faculty of Applied Science, Macao Polytechnic University, Macao SAR, China（澳门理工学院应用科学学院AIDD）；School of Computer Software, College of Intelligence and Computing, Tianjin University, Tianjin, China（天津大学计算机软件学院）
Shiraz University of Technology（设拉子科技大学）；National Brain Mapping Laboratory（国家脑图谱实验室）
Perineural invasion (PNI) by malignant tumor cells has been reported as an independent indicator of poor prognosis in various cancers. Assessment of PNI in small nerves on glass slides is a labor-intensive task. In this study, we propose an algorithm to detect the perineural invasions in colon, prostate, and pancreas cancers based on a convolutional neural network (CNN).
