Self-Supervised Learning for Organs At Risk and Tumor Segmentation with Uncertainty Quantification
University of Central Florida（中佛罗里达大学）；Northwestern University（西北大学）；KnowledgeVis LLC（KnowledgeVis有限责任公司）；Orlando Health（奥兰多健康）；University of Chicago（芝加哥大学）
Ilkin Isler, Debesh Jha, Curtis Lisle, Justin Rineer, Patrick Kelly, Bulent Aydogan, Mohamed Abazeed, Damla Turgut, Ulas Bagci
In this study, our goal is to show the impact of self-supervised pre-training of transformers for organ at risk (OAR) and tumor segmentation as compared to costly fully-supervised learning. The proposed algorithm is called Monte Carlo Transformer based U-Net (MC-Swin-U). Unlike many other available models, our approach presents uncertainty quantification with Monte Carlo dropout strategy while generating its voxel-wise prediction. We test and validate the proposed model on both public and one private datasets and evaluate the gross tumor volume (GTV) as well as nearby risky organs' boundaries. We show that self-supervised pre-training approach improves the segmentation scores significantly while providing additional benefits for avoiding large-scale annotation costs.
IBM Research - Europe（IBM欧洲研究院）；University College Dublin（都柏林大学学院）；Royal College of Surgeons（皇家外科医学院）
Intra-operative identification of malignant versus benign or healthy tissue is a major challenge in fluorescence guided cancer surgery. We propose a perfusion quantification method for computer-aided interpretation of subtle differences in dynamic perfusion patterns which can be used to distinguish between normal tissue and benign or malignant tumors intra-operatively in real-time by using multispectral endoscopic videos. The method exploits the fact that vasculature arising from cancer angiogenesis gives tumors differing perfusion patterns from the surrounding tissue, and defines a signature …
Anisotropic Hybrid Networks for liver tumor segmentation with uncertainty quantification
Univ. Grenoble Alpes（格勒诺布尔大学）；Inserm（法国国家健康与医学研究院）；Grenoble Institut Neurosciences（格勒诺布尔神经科学研究所）；Pixyl（Pixyl公司）；Research and Development Laboratory（研发实验室）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔综合理工学院）；LJK（格勒诺布尔大学联合实验室（注：因LJK无明确对应中文通用名称，按原文保留英文缩写））
Northwestern University（西北大学）；University of Chicago（芝加哥大学）
Head and Neck (H\&N) organ-at-risk (OAR) and tumor segmentations are essential components of radiation therapy planning. The varying anatomic locations and dimensions of H\&N nodal Gross Tumor Volumes (GTVn) and H\&N primary gross tumor volume (GTVp) are difficult to obtain due to lack of accurate and reliable delineation methods. The downstream effect of incorrect segmentation can result in unnecessary irradiation of normal organs. Towards a fully automated radiation therapy planning algorithm, we explore the efficacy of multi-scale fusion based deep learning architectures for accurately segm…
Southeast University（东南大学）；Swinburne University of Technology（斯威本科技大学）；Communication University of China（中国传媒大学）
Precise segmentation of organs and tumors plays a crucial role in clinical applications. It is a challenging task due to the irregular shapes and various sizes of organs and tumors as well as the significant class imbalance between the anatomy of interest (AOI) and the background region. In addition, in most situation tumors and normal organs often overlap in medical images, but current approaches fail to delineate both tumors and organs accurately. To tackle such challenges, we propose a position-guided deformable UNet, namely PGD-UNet, which exploits the spatial deformation capabilities of d…
Non-invasive techniques such as magnetic resonance imaging (MRI) are widely employed in brain tumor diagnostics. However, manual segmentation of brain tumors from 3D MRI volumes is a time-consuming task that requires trained expert radiologists. Due to the subjectivity of manual segmentation, there is low inter-rater reliability which can result in diagnostic discrepancies. As the success of many brain tumor treatments depends on early intervention, early detection is paramount. In this context, a fully automated segmentation method for brain tumor segmentation is necessary as an efficient and…
Department of Science and Technology, Linkoping University（林雪平大学科技学院）；Sectra AB（Sectra公司）；Center for Medical Image Science and Visualization, Linkoping University（林雪平大学医学影像科学与可视化中心）
The scarcity of labeled data is a major bottleneck for developing accurate and robust deep learning-based models for histopathology applications. The problem is notably prominent for the task of metastasis detection in lymph nodes, due to the tissue's low tumor-to-non-tumor ratio, resulting in labor- and time-intensive annotation processes for the pathologists. This work explores alternatives on how to augment the training data for colon carcinoma metastasis detection when there is limited or no representation of the target domain. Through an exhaustive study of cross-validated experiments wit…
Polytechnique Montreal（蒙特利尔理工学院）；Mila, Quebec AI Institute（米拉魁北克人工智能研究所）；Beijing Tiantan Hospital, Capital Medical University（首都医科大学附属北京天坛医院）；Université de Montréal（蒙特利尔大学）
Spinal cord tumors lead to neurological morbidity and mortality. Being able to obtain morphometric quantification (size, location, growth rate) of the tumor, edema, and cavity can result in improved monitoring and treatment planning. Such quantification requires the segmentation of these structures into three separate classes. However, manual segmentation of 3-dimensional structures is time-consuming and tedious, motivating the development of automated methods. Here, we tailor a model adapted to the spinal cord tumor segmentation task. Data were obtained from 343 patients using gadolinium-enha…
CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Johns Hopkins University（约翰斯·霍普金斯大学）；Hupan Lab（湖畔实验室）；Sun Yat-sen University Cancer Center（中山大学肿瘤防治中心）；Sichuan Cancer Hospital（四川省肿瘤医院）；The First Affiliated Hospital of Zhejiang University（浙江大学医学院附属第一医院）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；Guangdong Provincial People’s Hospital（广东省人民医院）
Breast tumor segmentation provides accurate tumor boundary, and serves as a key step toward further cancer quantification. Although deep learning-based approaches have been proposed and achieved promising results, existing approaches have difficulty in detecting small breast tumors. The capacity to detecting small tumors is particularly important in finding early stage cancers using computer-aided diagnosis (CAD) systems. In this paper, we propose a novel deep learning architecture called Small Tumor-Aware Network (STAN), to improve the performance of segmenting tumors with different size. The…
Automatic segmentation of brain glioma from multimodal MRI scans plays a key role in clinical trials and practice. Unfortunately, manual segmentation is very challenging, time-consuming, costly, and often inaccurate despite human expertise due to the high variance and high uncertainty in the human annotations. In the present work, we develop an end-to-end deep-learning-based segmentation method using a multi-decoder architecture by jointly learning three separate sub-problems using a partly shared encoder. We also propose to apply smoothing methods to the input images to generate denoised vers…
CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
City University of Hong Kong（香港城市大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Vanderbilt University（范德堡大学）；Chinese University of Hong Kong（香港中文大学）；CUHK Shenzhen Research Institute（香港中文大学深圳研究院）；NVIDIA（英伟达公司）
Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography
City University of Hong Kong（香港城市大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of California, San Francisco（加州大学旧金山分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Chinese University of Hong Kong（香港中文大学）；NVIDIA（英伟达）
PAII Inc.（PAII公司）；Chang Gung Memorial Hospital（长庚纪念医院）；Ping An Technology（平安科技）
Clinical target volume (CTV) delineation from radiotherapy computed tomography (RTCT) images is used to define the treatment areas containing the gross tumor volume (GTV) and/or sub-clinical malignant disease for radiotherapy (RT). High intra- and inter-user variability makes this a particularly difficult task for esophageal cancer. This motivates automated solutions, which is the aim of our work. Because CTV delineation is highly context-dependent--it must encompass the GTV and regional lymph nodes (LNs) while also avoiding excessive exposure to the organs at risk (OARs)--we formulate it as a…
School of Biomedical Engineering（生物医学工程学院）；Universidad Autónoma de Ciudad Juárez（华雷斯自治大学）
The brain is a complex organ controlling cognitive process and physical functions. Tumors in the brain are accelerated cell growths affecting the normal function and processes in the brain. MRI scans provides detailed images of the body being one of the most common tests to diagnose brain tumors. The process of segmentation of brain tumors from magnetic resonance imaging can provide a valuable guide for diagnosis, treatment planning and prediction of results. Here we consider the problem brain tumor segmentation using a Deep learning architecture for use in tumor segmentation. Although the pro…
UiT The Arctic University of Norway（挪威北极大学）；University Hospital of North Norway（北挪威大学医院）
Increased levels of tumor infiltrating lymphocytes (TILs) in cancer tissue indicate favourable outcomes in many types of cancer. Manual quantification of immune cells is inaccurate and time consuming for pathologists. Our aim is to leverage a computational solution to automatically quantify TILs in whole slide images (WSIs) of standard diagnostic haematoxylin and eosin stained sections (H&E slides) from lung cancer patients. Our approach is to transfer an open source machine learning method for segmentation and classification of nuclei in H&E slides trained on public data to TIL quantification…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Breast tumor segmentation is a critical task in computer-aided diagnosis (CAD) systems for breast cancer detection because accurate tumor size, shape and location are important for further tumor quantification and classification. However, segmenting small tumors in ultrasound images is challenging, due to the speckle noise, varying tumor shapes and sizes among patients, and the existence of tumor-like image regions. Recently, deep learning-based approaches have achieved great success for biomedical image analysis, but current state-of-the-art approaches achieve poor performance for segmenting …
National Institutes of Health（美国国立卫生研究院）；National Institute of Diabetes and Digestive and Kidney Diseases（国家糖尿病、消化和肾脏疾病研究所）
Finding small lesions is very challenging due to lack of noticeable features, severe class imbalance, as well as the size itself. One approach to improve small lesion segmentation is to reduce the region of interest and inspect it at a higher sensitivity rather than performing it for the entire region. It is usually implemented as sequential or joint segmentation of organ and lesion, which requires additional supervision on organ segmentation. Instead, we propose to utilize an intensity distribution of a target lesion at no additional labeling cost to effectively separate regions where the les…
With the development of image segmentation in computer vision, biomedical image segmentation have achieved remarkable progress on brain tumor segmentation and Organ At Risk (OAR) segmentation. However, most of the research only uses single modality such as Computed Tomography (CT) scans while in real world scenario doctors often use multiple modalities to get more accurate result. To better leverage different modalities, we have collected a large dataset consists of 136 cases with CT and MR images which diagnosed with nasopharyngeal cancer. In this paper, we propose to use Generative Adversari…
Xiamen University（厦门大学）；The Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；The Hong Kong Polytechnic University（香港理工大学）；Manteia Technologies Co.,Ltd（Manteia科技有限公司）
Automatic delineation of organ-at-risk (OAR) and gross-tumor-volume (GTV) is of great significance for radiotherapy planning. However, it is a challenging task to learn powerful representations for accurate delineation under limited pixel (voxel)-wise annotations. Contrastive learning at pixel-level can alleviate the dependency on annotations by learning dense representations from unlabeled data. Recent studies in this direction design various contrastive losses on the feature maps, to yield discriminative features for each pixel in the map. However, pixels in the same map inevitably share sem…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Brain tumor segmentation is essential for the diagnosis and prognosis of patients with gliomas. The brain tumor segmentation challenge has continued to provide a great source of data to develop automatic algorithms to perform the task. This paper describes our contribution to the 2021 competition. We developed our methods based on nn-UNet, the winning entry of last year competition. We experimented with several modifications, including using a larger network, replacing batch normalization with group normalization, and utilizing axial attention in the decoder. Internal 5-fold cross validation a…
SegRap2023: A Benchmark of Organs-at-Risk and Gross Tumor Volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma
University of Electronic Science and Technology of China（电子科技大学）；Sichuan Cancer Hospital & Institute（四川省肿瘤医院（研究所））；Shanghai AI Lab（上海人工智能实验室）；Canon Medical Systems (China) Co. Ltd.（佳能医疗系统（中国）有限公司）；Stockholm University（斯德哥尔摩大学）；KTH（瑞典皇家理工学院）；Northwestern Polytechnical University（西北工业大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）；Ping An Insurance Company of China（中国平安保险公司）
Developing an effective liver and liver tumor segmentation model from CT scans is very important for the success of liver cancer diagnosis, surgical planning and cancer treatment. In this work, we propose a two-stage framework for 2D liver and tumor segmentation. The first stage is a coarse liver segmentation network, while the second stage is an edge enhanced network (E$^2$Net) for more accurate liver and tumor segmentation. E$^2$Net explicitly models complementary objects (liver and tumor) and their edge information within the network to preserve the organ and lesion boundaries. We introduce…
Shiraz University of Technology（设拉子科技大学）；National Brain Mapping Laboratory（国家脑图谱实验室）
Perineural invasion (PNI) by malignant tumor cells has been reported as an independent indicator of poor prognosis in various cancers. Assessment of PNI in small nerves on glass slides is a labor-intensive task. In this study, we propose an algorithm to detect the perineural invasions in colon, prostate, and pancreas cancers based on a convolutional neural network (CNN).
Automatic segmentation of tumor lesions is a critical initial processing step for quantitative PET/CT analysis. However, numerous tumor lesion with different shapes, sizes, and uptake intensity may be distributed in different anatomical contexts throughout the body, and there is also significant uptake in healthy organs. Therefore, building a systemic PET/CT tumor lesion segmentation model is a challenging task. In this paper, we propose a novel training strategy to build deep learning models capable of systemic tumor segmentation. Our method is validated on the training set of the AutoPET 202…
Mansoura University（曼苏尔大学）；South Valley University（南河谷大学）；Higher Institute of Engineering and Technology（高等工程技术学院）
The liver is one of the most critical metabolic organs in vertebrates due to its vital functions in the human body, such as detoxification of the blood from waste products and medications. Liver diseases due to liver tumors are one of the most common mortality reasons around the globe. Hence, detecting liver tumors in the early stages of tumor development is highly required as a critical part of medical treatment. Many imaging modalities can be used as aiding tools to detect liver tumors. Computed tomography (CT) is the most used imaging modality for soft tissue organs such as the liver. This …
NVIDIA（英伟达）；University of California, Los Angeles（加州大学洛杉矶分校）
Automated segmentation of kidneys and kidney tumors is an important step in quantifying the tumor's morphometrical details to monitor the progression of the disease and accurately compare decisions regarding the kidney tumor treatment. Manual delineation techniques are often tedious, error-prone and require expert knowledge for creating unambiguous representation of kidneys and kidney tumors segmentation. In this work, we propose an end-to-end boundary aware fully Convolutional Neural Networks (CNNs) for reliable kidney and kidney tumor semantic segmentation from arterial phase abdominal 3D CT…
Stanford University（斯坦福大学）；Varian Medical Systems（瓦里安医疗系统公司）；Huiyihuiying Medical Co（慧医慧影医疗公司）；Google Inc（谷歌公司）
Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra-/inter- operator variations. While various algorithms for delineating organ-at-risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U-Net has gained increasing popularity recently for ima…
The advancements in deep learning technologies have produced immense contributions to biomedical image analysis applications. With breast cancer being the common deadliest disease among women, early detection is the key means to improve survivability. Medical imaging like ultrasound presents an excellent visual representation of the functioning of the organs; however, for any radiologist analysing such scans is challenging and time consuming which delays the diagnosis process. Although various deep learning based approaches are proposed that achieved promising results, the present article intr…
Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation
Université de technologie de Compiègne（贡比涅技术大学）；CNRS（法国国家科学研究中心）；Université de Rouen Normandie（鲁昂诺曼底大学）；Centre Henri Becquerel（亨利·贝克勒尔中心）；Institut universitaire de France（法国大学研究院）
Diff4MMLiTS: Advanced Multimodal Liver Tumor Segmentation via Diffusion-Based Image Synthesis and Alignment
Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China（电子与电气工程系，南方科技大学，深圳，中国）；Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China（电气与电子工程系，香港大学，香港特别行政区，中国）；Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing, China（放射科，中大医院，医学院，东南大学，南京，中国）；Jiaxing Research Institute, Southern University of Science and Technology, Jiaxing, China（嘉兴研究所，南方科技大学，嘉兴，中国）
Medical University of Vienna（维也纳医科大学）；Intel（英特尔公司）；Lobachevsky State University（罗巴切夫斯基国立大学）
The segmentation of brain tumors in multimodal MRIs is one of the most challenging tasks in medical image analysis. The recent state of the art algorithms solving this task is based on machine learning approaches and deep learning in particular. The amount of data used for training such models and its variability is a keystone for building an algorithm with high representation power. In this paper, we study the relationship between the performance of the model and the amount of data employed during the training process. On the example of brain tumor segmentation challenge, we compare the model…
Analog In-Memory Computing with Uncertainty Quantification for Efficient Edge-based Medical Imaging Segmentation
École nationale Supérieure d’Informatique（国立高等计算机学院）；IBM Research Europe（IBM欧洲研究院）；Yorktown High school（约克镇高中）；IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）
The accurate automatic segmentation of gliomas and its intra-tumoral structures is important not only for treatment planning but also for follow-up evaluations. Several methods based on 2D and 3D Deep Neural Networks (DNN) have been developed to segment brain tumors and to classify different categories of tumors from different MRI modalities. However, these networks are often black-box models and do not provide any evidence regarding the process they take to perform this task. Increasing transparency and interpretability of such deep learning techniques are necessary for the complete integrati…
Deep Learning for Automated Quantification of Tumor-Associated Macrophages from H&E-Stained Slides in Diffuse Large B-Cell Lymphoma
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究所）；University of Sharjah（沙迦大学）；Tokai University School of Medicine（东海大学医学院）；National Research University Higher School of Economics（国立研究高等经济大学）；Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）；Vavilov Institute of General Genetics（瓦维洛夫普通遗传学研究所）；Institute of Plant and Animal Ecology Ural Branch of Russian Academy of Sciences（俄罗斯科学院乌拉尔分校动植物生态学研究所）
Convolutional neural networks (CNNs) have achieved remarkable success in automatically segmenting organs or lesions on 3D medical images. Recently, vision transformer networks have exhibited exceptional performance in 2D image classification tasks. Compared with CNNs, transformer networks have an appealing advantage of extracting long-range features due to their self-attention algorithm. Therefore, we propose a CNN-Transformer combined model, called BiTr-Unet, with specific modifications for brain tumor segmentation on multi-modal MRI scans. Our BiTr-Unet achieves good performance on the BraTS…
Imperial College London（帝国理工学院）；TECNALIA, Basque Research & Technology Alliance（巴斯克研究与技术联盟TECNALIA）；University of the Basque Country（巴斯克大学）；Royal Brompton Hospital（皇家布朗普顿医院）
Automatic segmentation of multiple organs and tumors from 3D medical images such as magnetic resonance imaging (MRI) and computed tomography (CT) scans using deep learning methods can aid in diagnosing and treating cancer. However, organs often overlap and are complexly connected, characterized by extensive anatomical variation and low contrast. In addition, the diversity of tumor shape, location, and appearance, coupled with the dominance of background voxels, makes accurate 3D medical image segmentation difficult. In this paper, a novel large-kernel (LK) attention module is proposed to addre…
University of Sydney（悉尼大学）；Shanghai Jiao Tong University（上海交通大学）
Multi-modality Fluorodeoxyglucose (FDG) positron emission tomography / computed tomography (PET/CT) has been routinely used in the assessment of common cancers, such as lung cancer, lymphoma, and melanoma. This is mainly attributed to the fact that PET/CT combines the high sensitivity for tumor detection of PET and anatomical information from CT. In PET/CT image assessment, automatic tumor segmentation is an important step, and in recent years, deep learning based methods have become the state-of-the-art. Unfortunately, existing methods tend to over-segment the tumor regions and include region…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Incorporating human domain knowledge for breast tumor diagnosis is challenging, since shape, boundary, curvature, intensity, or other common medical priors vary significantly across patients and cannot be employed. This work proposes a new approach for integrating visual saliency into a deep learning model for breast tumor segmentation in ultrasound images. Visual saliency refers to image maps containing regions that are more likely to attract radiologists visual attention. The proposed approach introduces attention blocks into a U-Net architecture, and learns feature representations that prio…
