Automatic segmentation of head and neck tumors plays an important role in radiomics analysis. In this short paper, we propose an automatic segmentation method for head and neck tumors from PET and CT images based on the combination of convolutional neural networks (CNNs) and hybrid active contours. Specifically, we first introduce a multi-channel 3D U-Net to segment the tumor with the concatenated PET and CT images. Then, we estimate the segmentation uncertainty by model ensembles and define a segmentation quality score to select the cases with high uncertainties. Finally, we develop a hybrid active contour model to refine the high uncertainty cases. Our method ranked second place in the MICCAI 2020 HECKTOR challenge with average Dice Similarity Coefficient, precision, and recall of 0.752, 0.838, and 0.717, respectively.
Head and neck tumor segmentation challenge (HECKTOR) 2022 offers a platform for researchers to compare their solutions to segmentation of tumors and lymph nodes from 3D CT and PET images. In this work, we describe our solution to HECKTOR 2022 segmentation task. We re-sample all images to a common resolution, crop around head and neck region, and train SegResNet semantic segmentation network from MONAI. We use 5-fold cross validation to select best model checkpoints. The final submission is an ensemble of 15 models from 3 runs. Our solution (team name NVAUTO) achieves the 1st place on the HECKT…
LaTIM, INSERM, UMR 1101, University Brest（拉蒂姆研究所、法国国家健康与医学研究院、布雷斯特大学）
Development of robust and accurate fully automated methods for medical image segmentation is crucial in clinical practice and radiomics studies. In this work, we contributed an automated approach for Head and Neck (H&N) primary tumor segmentation in combined positron emission tomography / computed tomography (PET/CT) images in the context of the MICCAI 2020 Head and Neck Tumor segmentation challenge (HECKTOR). Our model was designed on the U-Net architecture with residual layers and supplemented with Squeeze-and-Excitation Normalization. The described method achieved competitive results in cro…
In radiotherapy planning, manual contouring is labor-intensive and time-consuming. Accurate and robust automated segmentation models improve the efficiency and treatment outcome. We aim to develop a novel hybrid deep learning approach, combining convolutional neural networks (CNNs) and the self-attention mechanism, for rapid and accurate multi-organ segmentation on head and neck computed tomography (CT) images. Head and neck CT images with manual contours of 115 patients were retrospectively collected and used. We set the training/validation/testing ratio to 81/9/25 and used the 10-fold cross-…
An automatic segmentation algorithm for delineation of the gross tumour volume and pathologic lymph nodes of head and neck cancers in PET/CT images is described. The proposed algorithm is based on a convolutional neural network using the U-Net architecture. Several model hyperparameters were explored and the model performance in terms of the Dice similarity coefficient was validated on images from 15 patients. A separate test set consisting of images from 40 patients was used to assess the generalisability of the algorithm. The performance on the test set showed close-to-oncologist level delin…
HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland（瑞士西部应用科学与艺术大学瓦莱州分校）；Centre Hospitalier Universitaire Vaudois (CHUV)（沃州大学中心医院）；Centre Hospitalier Universitaire de Poitiers (CHUP)（普瓦捷大学中心医院）；Univ Brest（布雷斯特大学）；INSERM（法国国家健康与医学研究院）；LaTIM（医学信息处理实验室）；Cleveland Clinic Foundation（克利夫兰医学中心基金会）；University of Sherbrooke（舍布鲁克大学）
This paper presents an overview of the second edition of the HEad and neCK TumOR (HECKTOR) challenge, organized as a satellite event of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021. The challenge is composed of three tasks related to the automatic analysis of PET/CT images for patients with Head and Neck cancer (H&N), focusing on the oropharynx region. Task 1 is the automatic segmentation of H&N primary Gross Tumor Volume (GTVt) in FDG-PET/CT images. Task 2 is the automatic prediction of Progression Free Survival (PFS) from the s…
Leiden University Medical Center（莱顿大学医学中心）；TU Delft（代尔夫特理工大学）
Deep learning models for organ contouring in radiotherapy are poised for clinical usage, but currently, there exist few tools for automated quality assessment (QA) of the predicted contours. Using Bayesian models and their associated uncertainty, one can potentially automate the process of detecting inaccurate predictions. We investigate two Bayesian models for auto-contouring, DropOut and FlipOut, using a quantitative measure - expected calibration error (ECE) and a qualitative measure - region-based accuracy-vs-uncertainty (R-AvU) graphs. It is well understood that a model should have low EC…
In medical imaging, radiological scans of different modalities serve to enhance different sets of features for clinical diagnosis and treatment planning. This variety enriches the source information that could be used for outcome prediction. Deep learning methods are particularly well-suited for feature extraction from high-dimensional inputs such as images. In this work, we apply a CNN classification network augmented with a FCN preprocessor sub-network to a public TCIA head and neck cancer dataset. The training goal is survival prediction of radiotherapy cases based on pre-treatment FDG PET-…
SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images
Philips Research North America（飞利浦北美研究院）；Massachusetts General Hospital/Harvard Medical School（麻省总医院/哈佛医学院）；The Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital/Harvard Medical School（麻省总医院/哈佛医学院高级医学计算与分析中心）
BC Cancer Research Institute（不列颠哥伦比亚癌症研究所）；University of British Columbia（不列颠哥伦比亚大学）；Simon Fraser University（西蒙弗雷泽大学）
We utilized a 3D nnU-Net model with residual layers supplemented by squeeze and excitation (SE) normalization for tumor segmentation from PET/CT images provided by the Head and Neck Tumor segmentation chal-lenge (HECKTOR). Our proposed loss function incorporates the Unified Fo-cal and Mumford-Shah losses to take the advantage of distribution, region, and boundary-based loss functions. The results of leave-one-out-center-cross-validation performed on different centers showed a segmentation performance of 0.82 average Dice score (DSC) and 3.16 median Hausdorff Distance (HD), and our results on t…
Background: Deep learning (DL)-based head and neck lymph node level (HN_LNL) autodelineation is of high relevance to radiotherapy research and clinical treatment planning but still underinvestigated in academic literature. Methods: An expert-delineated cohort of 35 planning CTs was used for training of an nnU-net 3D-fullres/2D-ensemble model for autosegmentation of 20 different HN_LNL. A second cohort acquired at the same institution later in time served as the test set (n=20). In a completely blinded evaluation, 3 clinical experts rated the quality of DL autosegmentations in a head-to-head co…
Southern Medical University（南方医科大学）；LaTIM（拉蒂姆实验室）；INSERM（法国国家健康与医学研究院）；Univ Brest（布雷斯特大学）；Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院）
Automatic segmentation of head and neck cancer (HNC) tumors and lymph nodes plays a crucial role in the optimization treatment strategy and prognosis analysis. This study aims to employ nnU-Net for automatic segmentation and radiomics for recurrence-free survival (RFS) prediction using pretreatment PET/CT images in multi-center HNC cohort. A multi-center HNC dataset with 883 patients (524 patients for training, 359 for testing) was provided in HECKTOR 2022. A bounding box of the extended oropharyngeal region was retrieved for each patient with fixed size of 224 x 224 x 224 $mm^{3}$. Then 3D nn…
University of Freiburg（弗莱堡大学）；Medical Center – University of Freiburg（弗莱堡大学医学中心）；Faculty of Medicine, University of Freiburg（弗莱堡大学医学院）；German Cancer Consortium (DKTK)（德国癌症联盟）；Medical School of Nanjing University（南京大学医学院）；Affiliated Drum Tower Hospital（鼓楼医院）；Hannover Medical School（汉诺威医学院）；Institute for Surgical Pathology, Medical Center – University of Freiburg（弗莱堡大学医学中心外科病理研究所）
Accurate delineation of the intraprostatic gross tumour volume (GTV) is a prerequisite for treatment approaches in patients with primary prostate cancer (PCa). Prostate-specific membrane antigen positron emission tomography (PSMA-PET) may outperform MRI in GTV detection. However, visual GTV delineation underlies interobserver heterogeneity and is time consuming. The aim of this study was to develop a convolutional neural network (CNN) for automated segmentation of intraprostatic tumour (GTV-CNN) in PSMA-PET. Methods: The CNN (3D U-Net) was trained on [68Ga]PSMA-PET images of 152 patients from …
Background and Purpose: Radiomics features are used to identify disease types and predict therapy outcomes. However, how the radiomics features are different among different anatomical structures has never been investigated. Hence, we analyzed the radiomics features of 22 anatomical structures in the head and neck area in CT images. Furthermore, we studied whether CT radiomics can classify anatomical structures of the head and neck using unsupervised machine-learning techniques. Materials and methods: We obtained IMRT/VMAT treatment planning data from 36 patients treated for head and neck canc…
Cancer Hospital of University of the Chinese Academy of Sciences (Zhejiang Cancer Hospital)（中国科学院大学附属肿瘤医院（浙江省肿瘤医院））；Rutgers University（罗格斯大学）；The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）
Radiotherapy is a treatment where radiation is used to eliminate cancer cells. The delineation of organs-at-risk (OARs) is a vital step in radiotherapy treatment planning to avoid damage to healthy organs. For nasopharyngeal cancer, more than 20 OARs are needed to be precisely segmented in advance. The challenge of this task lies in complex anatomical structure, low-contrast organ contours, and the extremely imbalanced size between large and small organs. Common segmentation methods that treat them equally would generally lead to inaccurate small-organ labeling. We propose a novel two-stage de…
Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer
School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）；Royal Prince Alfred Hospital（皇家阿尔弗雷德王子医院）；Med-X Research Institute, Shanghai Jiao Tong University（上海交通大学Med-X研究院）
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Accurate prognosis of a tumor can help doctors provide a proper course of treatment and, therefore, save the lives of many. Traditional machine learning algorithms have been eminently useful in crafting prognostic models in the last few decades. Recently, deep learning algorithms have shown significant improvement when developing diagnosis and prognosis solutions to different healthcare problems. However, most of these solutions rely solely on either imaging or clinical data. Utilizing patient tabular data such as demographics and patient medical history alongside imaging data in a multimodal …
SimulaMet（SimulaMet研究院）；Oslo Metropolitan University（奥斯陆城市大学）；UiT The Arctic University of Norway（挪威北极大学）
Head and neck cancers are the fifth most common cancer worldwide, and recently, analysis of Positron Emission Tomography (PET) and Computed Tomography (CT) images has been proposed to identify patients with a prognosis. Even though the results look promising, more research is needed to further validate and improve the results. This paper presents the work done by team MLC for the 2022 version of the HECKTOR grand challenge held at MICCAI 2022. For Task 1, the automatic segmentation task, our approach was, in contrast to earlier solutions using 3D segmentation, to keep it as simple as possible …
Segmentation of head and neck (H\&N) tumours and prediction of patient outcome are crucial for patient's disease diagnosis and treatment monitoring. Current developments of robust deep learning models are hindered by the lack of large multi-centre, multi-modal data with quality annotations. The MICCAI 2021 HEad and neCK TumOR (HECKTOR) segmentation and outcome prediction challenge creates a platform for comparing segmentation methods of the primary gross target volume on fluoro-deoxyglucose (FDG)-PET and Computed Tomography images and prediction of progression-free survival in H\&N oropharynge…
Purpose: Organ-at-risk (OAR) delineation is a key step for cone-beam CT (CBCT) based adaptive radiotherapy planning that can be a time-consuming, labor-intensive, and subject-to-variability process. We aim to develop a fully automated approach aided by synthetic MRI for rapid and accurate CBCT multi-organ contouring in head-and-neck (HN) cancer patients. MRI has superb soft-tissue contrasts, while CBCT offers bony-structure contrasts. Using the complementary information provided by MRI and CBCT is expected to enable accurate multi-organ segmentation in HN cancer patients. In our proposed metho…
The University of Sydney（悉尼大学）；Shanghai Jiao Tong University（上海交通大学）
Outcome prediction is crucial for head and neck cancer patients as it can provide prognostic information for early treatment planning. Radiomics methods have been widely used for outcome prediction from medical images. However, these methods are limited by their reliance on intractable manual segmentation of tumor regions. Recently, deep learning methods have been proposed to perform end-to-end outcome prediction so as to remove the reliance on manual segmentation. Unfortunately, without segmentation masks, these methods will take the whole image as input, such that makes them difficult to foc…
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Cancer is one of the leading causes of death worldwide, and head and neck (H&N) cancer is amongst the most prevalent types. Positron emission tomography and computed tomography are used to detect, segment and quantify the tumor region. Clinically, tumor segmentation is extensively time-consuming and prone to error. Machine learning, and deep learning in particular, can assist to automate this process, yielding results as accurate as the results of a clinician. In this paper, we investigate a vision transformer-based method to automatically delineate H&N tumor, and compare its results to leadin…
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院同济医院）；Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions（深港脑科学创新研究院——深圳基础研究机构）；United Imaging Research Institute of Innovative Medical Equipment（联影创新医疗装备研究院）；Shenzhen Bay Laboratory（深圳湾实验室）
Achieving accurate and automated tumor segmentation plays an important role in both clinical practice and radiomics research. Segmentation in medicine is now often performed manually by experts, which is a laborious, expensive and error-prone task. Manual annotation relies heavily on the experience and knowledge of these experts. In addition, there is much intra- and interobserver variation. Therefore, it is of great significance to develop a method that can automatically segment tumor target regions. In this paper, we propose a deep learning segmentation method based on multimodal positron em…
Automatic segmentation of Organs at Risk in Head and Neck cancer patients from CT and MRI scans
Medical Physics Unit, Department of Oncology, McGill University（麦吉尔大学医学物理单位）；Montreal Institute for Learning Algorithms, Mila（蒙特利尔学习算法研究所）；Department of Computer Science, University of Calgary（卡尔加里大学计算机科学系）；University of British Columbia（不列颠哥伦比亚大学）；Division of Radiation Oncology, Department of Oncology, McGill University（麦吉尔大学放射肿瘤学部）；McGill University Health Centre（麦吉尔大学健康中心）；Lady Davis Institute for Medical Research, Jewish General Hospital（Lady Davis医学研究所，犹太通用医院）；Department of Diagnostic Radiology, McGill University（麦吉尔大学诊断放射学部）；Department of Radiology, University of Florida（佛罗里达大学放射学部）
Data Science Institute, University of British Columbia（不列颠哥伦比亚大学数据科学研究所）；BC Cancer Research Institute（不列颠哥伦比亚癌症研究所）；University of British Columbia（不列颠哥伦比亚大学）
Training computer-vision related algorithms on medical images for disease diagnosis or image segmentation is difficult in large part due to privacy concerns. For this reason, generative image models are highly sought after to facilitate data sharing. However, 3-D generative models are understudied, and investigation of their privacy leakage is needed. We introduce our 3-D generative model, Transversal GAN (TrGAN), using head & neck PET images which are conditioned on tumour masks as a case study. We define quantitative measures of image fidelity, utility and privacy for our model. These metric…
University of British Columbia Data Science Institute（不列颠哥伦比亚大学数据科学研究所）；BC Cancer Research Institute（不列颠哥伦比亚省癌症研究所）；University of British Columbia（不列颠哥伦比亚大学）
Training computer-vision related algorithms on medical images for disease diagnosis or image segmentation is difficult due to the lack of training data, labeled samples, and privacy concerns. For this reason, a robust generative method to create synthetic data is highly sought after. However, most three-dimensional image generators require additional image input or are extremely memory intensive. To address these issues we propose adapting video generation techniques for 3-D image generation. Using the temporal GAN (TGAN) architecture, we show we are able to generate realistic head and neck PE…
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH（VRVis虚拟现实与可视化研究有限公司）；Center for Medical Physics and Biomedical Engineering, Medical University of Vienna（维也纳医科大学医学物理与生物医学工程中心）；Center for Nuclear Medicine, Clinical Center of Serbia（塞尔维亚临床中心核医学中心）
Tumor segmentation in multimodal medical images has seen a growing trend towards deep learning based methods. Typically, studies dealing with this topic fuse multimodal image data to improve the tumor segmentation contour for a single imaging modality. However, they do not take into account that tumor characteristics are emphasized differently by each modality, which affects the tumor delineation. Thus, the tumor segmentation is modality- and task-dependent. This is especially the case for soft tissue sarcomas, where, due to necrotic tumor tissue, the segmentation differs vastly. Closing this …
University Medical Center Groningen（格罗宁根大学医学中心）；University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）
Tumor segmentation is a fundamental step for radiotherapy treatment planning. To define an accurate segmentation of the primary tumor (GTVp) of oropharyngeal cancer patients (OPC), simultaneous assessment of different image modalities is needed, and each image volume is explored slice-by-slice from different orientations. Moreover, the manual fixed boundary of segmentation neglects the spatial uncertainty known to occur in tumor delineation. This study proposes a novel automatic deep learning (DL) model to assist radiation oncologists in a slice-by-slice adaptive GTVp segmentation on registere…
GLOW-FDG: Generalized cancer LesiOn Whole-body segmentation model for $^{18}$F-FDG-PET/CT
Department of Radiation Oncology, University Hospital Zurich, University of Zurich, Zurich, Switzerland（苏黎世大学放射肿瘤科，苏黎世大学，苏黎世，瑞士）；Faculty of Medicine, University of Zurich, Zurich, Switzerland（苏黎世大学医学院，苏黎世，瑞士）；Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany（德国癌症研究中心（DKFZ）医学图像计算部，海德堡，德国）；Department of Radiation Oncology, Institut de Cancérologie de Lorraine, Vandœuvre-lès-Nancy, France（洛林癌症研究所放射肿瘤科，法国 Vandœuvre-lès-Nancy）；Department of Thoracic Surgery, University Hospital Zurich, University of Zurich, Zurich, Switzerland（苏黎世大学胸外科科，苏黎世大学，苏黎世，瑞士）；ETOP IBCSG Partners Foundation, Bern, Switzerland（ETOP IBCSG合作伙伴基金会，瑞士伯恩）；Department of Nuclear Medicine, University Hospital Zurich, University of Zurich, Zurich, Switzerland（苏黎世大学核医学科，苏黎世大学，苏黎世，瑞士）
University of Iowa（爱荷华大学）；Stanford University（斯坦福大学）
Non-small-cell lung cancer (NSCLC) represents approximately 80-85% of lung cancer diagnoses and is the leading cause of cancer-related death worldwide. Recent studies indicate that image-based radiomics features from positron emission tomography-computed tomography (PET/CT) images have predictive power on NSCLC outcomes. To this end, easily calculated functional features such as the maximum and the mean of standard uptake value (SUV) and total lesion glycolysis (TLG) are most commonly used for NSCLC prognostication, but their prognostic value remains controversial. Meanwhile, convolutional neu…
Adaptive radiotherapy (ART), especially online ART, effectively accounts for positioning errors and anatomical changes. One key component of online ART is accurately and efficiently delineating organs at risk (OARs) and targets on online images, such as CBCT, to meet the online demands of plan evaluation and adaptation. Deep learning (DL)-based automatic segmentation has gained great success in segmenting planning CT, but its applications to CBCT yielded inferior results due to the low image quality and limited available contour labels for training. To overcome these obstacles to online CBCT s…
