AttentionAnatomy: A unified framework for whole-body organs at risk segmentation using multiple partially annotated datasets
DeepVoxel Inc（深体素公司）；Shanghai General Hospital（上海市第一人民医院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）；University of California, Irvine（加州大学欧文分校）
Organs-at-risk (OAR) delineation in computed tomography (CT) is an important step in Radiation Therapy (RT) planning. Recently, deep learning based methods for OAR delineation have been proposed and applied in clinical practice for separate regions of the human body (head and neck, thorax, and abdomen). However, there are few researches regarding the end-to-end whole-body OARs delineation because the existing datasets are mostly partially or incompletely annotated for such task. In this paper, our proposed end-to-end convolutional neural network model, called \textbf{AttentionAnatomy}, can be jointly trained with three partially annotated datasets, segmenting OARs from whole body. Our main contributions are: 1) an attention module implicitly guided by body region label to modulate the segmentation branch output; 2) a prediction re-calibration operation, exploiting prior information of the input images, to handle partial-annotation(HPA) problem; 3) a new hybrid loss function combining batch Dice loss and spatially balanced focal loss to alleviate the organ size imbalance problem. Experimental results of our proposed framework presented significant improvements in both Sørensen-Dice coefficient (DSC) and 95\% Hausdorff distance compared to the baseline model.
SenseTime Research（商汤科技研究院）；School of Information Science and Engineering at Shandong Normal University（山东师范大学信息科学与工程学院）；CUHK-SenseTime Joint Laboratory, The Chinese University of Hong Kong（香港中文大学-商汤科技联合实验室）
Multi-organ segmentation has extensive applications in many clinical applications. To segment multiple organs of interest, it is generally quite difficult to collect full annotations of all the organs on the same images, as some medical centers might only annotate a portion of the organs due to their own clinical practice. In most scenarios, one might obtain annotations of a single or a few organs from one training set, and obtain annotations of the the other organs from another set of training images. Existing approaches mostly train and deploy a single model for each subset of organs, which …
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（上海交通大学媒体网络协同创新中心）
Annotating multiple organs in 3D medical images is time-consuming and costly. Meanwhile, there exist many single-organ datasets with one specific organ annotated. This paper investigates how to learn a multi-organ segmentation model leveraging a set of binary-labeled datasets. A novel Multi-teacher Single-student Knowledge Distillation (MS-KD) framework is proposed, where the teacher models are pre-trained single-organ segmentation networks, and the student model is a multi-organ segmentation network. Considering that each teacher focuses on different organs, a region-based supervision method,…
ADPv2: A Hierarchical Histological Tissue Type-Annotated Dataset for Potential Biomarker Discovery of Colorectal Disease
Department of Computer Science \& Software Engineering, Concordia University, 2155 Guy St, Montreal, QC H3H 2L9, Canada；Department of Electrical \& Computer Engineering, University of Toronto, 10 King's College Rd, Toronto, ON M5S 3G8, Canada；Department of Chemistry \& Biology, Toronto Metropolitan University, 350 Victoria St. Toronto, ON M5B 2K3, Canada；Department of Medicine, Université de Montréal, Pavillon Roger-Gaudry, 2900 Edouard Montpetit Blvd, Montreal, QC H3T 1J4, Canada；Axe Cancer, Centre de recherche du CHUM, 900 Saint-Denis St, Montréal, QC H2X 0A9, Canada；Anatomic Pathology, Sunnybrook Health Sciences Centre, 2075 Bayview Ave, Toronto, ON M4N 3M5, Canada；Department of Laboratory Medicine \& Pathobiology, University of Toronto, Simcoe Hall, 1 King's College Circle, Toronto, ON M5S 3K3, Canada；Department of Pathology \& Molecular Medicine, Queen’s University, 88 Stuart Street Queen's University Kingston, ON K7L 3N6 Canada；Department of Pathology \& Molecular Medicine, Université de Montréal, 2900 Édouard-Montpetit Blvd, Montréal, QC H3T 1J4, Canada
The First Affiliated Hospital of Anhui Medical University（安徽医科大学第一附属医院）；Huawei Cloud Computing Technology Co. Ltd（华为云计算技术有限公司）；University of Science and Technology of China（中国科学技术大学）
There exists a large number of datasets for organ segmentation, which are partially annotated and sequentially constructed. A typical dataset is constructed at a certain time by curating medical images and annotating the organs of interest. In other words, new datasets with annotations of new organ categories are built over time. To unleash the potential behind these partially labeled, sequentially-constructed datasets, we propose to incrementally learn a multi-organ segmentation model. In each incremental learning (IL) stage, we lose the access to previous data and annotations, whose knowledg…
Supervised learning is often used to count objects in images, but for counting small, densely located objects, the required image annotations are burdensome to collect. Counting plant organs for image-based plant phenotyping falls within this category. Object counting in plant images is further challenged by having plant image datasets with significant domain shift due to different experimental conditions, e.g. applying an annotated dataset of indoor plant images for use on outdoor images, or on a different plant species. In this paper, we propose a domain-adversarial learning approach for dom…
University of Electronic Science and Technology of China（电子科技大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Sichuan Cancer Hospital and Institute（四川省肿瘤医院（四川省肿瘤研究所））；Sichuan Cancer Center（四川省癌症中心）；Sichuan University（四川大学）；West China Hospital（华西医院）；Johns Hopkins University（约翰斯·霍普金斯大学）；SenseTime Research（商汤科技研究院）；Rutgers University（罗格斯大学）
Whole abdominal organ segmentation is important in diagnosing abdomen lesions, radiotherapy, and follow-up. However, oncologists' delineating all abdominal organs from 3D volumes is time-consuming and very expensive. Deep learning-based medical image segmentation has shown the potential to reduce manual delineation efforts, but it still requires a large-scale fine annotated dataset for training, and there is a lack of large-scale datasets covering the whole abdomen region with accurate and detailed annotations for the whole abdominal organ segmentation. In this work, we establish a new large-s…
RPTU Kaiserslautern-Landau（莱茵兰-普法尔茨州凯泽斯劳滕-兰道应用技术大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Heidelberg University（海德堡大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
Morphological atlases are an important tool in organismal studies, and modern high-throughput Computed Tomography (CT) facilities can produce hundreds of full-body high-resolution volumetric images of organisms. However, creating an atlas from these volumes requires accurate organ segmentation. In the last decade, machine learning approaches have achieved incredible results in image segmentation tasks, but they require large amounts of annotated data for training. In this paper, we propose a self-training framework for multi-organ segmentation in tomographic images of Medaka fish. We utilize t…
Global Rice Multi-Class Segmentation Dataset (RiceSEG): A Comprehensive and Diverse High-Resolution RGB-Annotated Images for the Development and Benchmarking of Rice Segmentation Algorithms
Nanjing Agricultural University（南京农业大学）；The University of Tokyo（东京大学）；Professor Jayashankar Telangana Agricultural State University（贾亚尚卡尔·特伦甘纳农业州立大学）；Kyoto University（京都大学）；Huazhong University of Science and Technology（华中科技大学）；Huazhong Agricultural University（华中农业大学）；Chinese Academy of Agricultural Sciences（中国农业科学院）；Chinese Academy of Science（中国科学院）；Shenyang Agricultural University（沈阳农业大学）；Jilin Academy of Agricultural Sciences（吉林省农业科学院）；Yuan Long Ping High-Tech Agriculture Co., Ltd.（袁隆平高科技农业有限公司）
Biomedical Image Analysis Group, Imperial College London（帝国理工学院生物医学图像分析组）；Comprehensive Cancer Imaging Centre, Hammersmith Hospital, Imperial College London（帝国理工学院哈默史密斯医院综合癌症成像中心）；The Royal Marsden NHS Foundation Trust, London（伦敦皇家马斯登国民保健服务基金会信托）
The variations in multi-center data in medical imaging studies have brought the necessity of domain adaptation. Despite the advancement of machine learning in automatic segmentation, performance often degrades when algorithms are applied on new data acquired from different scanners or sequences than the training data. Manual annotation is costly and time consuming if it has to be carried out for every new target domain. In this work, we investigate automatic selection of suitable subjects to be annotated for supervised domain adaptation using the concept of reverse classification accuracy (RCA…
MultiOrg: A Multi-rater Organoid-detection Dataset
Helmholtz AI（亥姆霍兹AI）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；Institute of Lung Health and Immunity (LHI)（肺部健康与免疫研究所）；Comprehensive Pneumology Center (CPC)（综合肺病学中心）；German Center for Lung Research (DZL)（德国肺部研究中心）；Technical University of Munich（慕尼黑工业大学）；Philipps-University（菲利普斯大学）
Technical University of Munich（慕尼黑工业大学）；Ulm University（乌尔姆大学）
The lack of sufficient annotated image data is a common issue in medical image segmentation. For some organs and densities, the annotation may be scarce, leading to poor model training convergence, while other organs have plenty of annotated data. In this work, we present MetaMedSeg, a gradient-based meta-learning algorithm that redefines the meta-learning task for the volumetric medical data with the goal to capture the variety between the slices. We also explore different weighting schemes for gradients aggregation, arguing that different tasks might have different complexity, and hence, con…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Non-contrast computed tomography (NCCT) is commonly acquired for lung cancer screening, assessment of general abdominal pain or suspected renal stones, trauma evaluation, and many other indications. However, the absence of contrast limits distinguishing organ in-between boundaries. In this paper, we propose a novel unsupervised approach that leverages pairwise contrast-enhanced CT (CECT) context to compute non-contrast segmentation without ground-truth label. Unlike generative adversarial approaches, we compute the pairwise morphological context with CECT to provide teacher guidance instead of…
University of New South Wales（新南威尔士大学）；Prince of Wales Hospital（威尔士亲王医院）
Computer aided diagnostics often requires analysis of a region of interest (ROI) within a radiology scan, and the ROI may be an organ or a suborgan. Although deep learning algorithms have the ability to outperform other methods, they rely on the availability of a large amount of annotated data. Motivated by the need to address this limitation, an approach to localisation and detection of multiple organs based on supervised and semi-supervised learning is presented here. It draws upon previous work by the authors on localising the thoracic and lumbar spine region in CT images. The method genera…
Sparsity of annotated data is a major limitation in medical image processing tasks such as registration. Registered multimodal image data are essential for the diagnosis of medical conditions and the success of interventional medical procedures. To overcome the shortage of data, we present a method that allows the generation of annotated multimodal 4D datasets. We use a CycleGAN network architecture to generate multimodal synthetic data from the 4D extended cardiac-torso (XCAT) phantom and real patient data. Organ masks are provided by the XCAT phantom, therefore the generated dataset can serv…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Performing coarse-to-fine abdominal multi-organ segmentation facilitates to extract high-resolution segmentation minimizing the lost of spatial contextual information. However, current coarse-to-refine approaches require a significant number of models to perform single organ refine segmentation corresponding to the extracted organ region of interest (ROI). We propose a coarse-to-fine pipeline, which starts from the extraction of the global prior context of multiple organs from 3D volumes using a low-resolution coarse network, followed by a fine phase that uses a single refined model to segment…
University of Alicante（阿利坎特大学）；Hospital San Juan de Alicante（阿利坎特圣胡安医院）；Regional Ministry of Health, Valencia（瓦伦西亚地区卫生部）
We present a labeled large-scale, high resolution chest x-ray dataset for the automated exploration of medical images along with their associated reports. This dataset includes more than 160,000 images obtained from 67,000 patients that were interpreted and reported by radiologists at Hospital San Juan Hospital (Spain) from 2009 to 2017, covering six different position views and additional information on image acquisition and patient demography. The reports were labeled with 174 different radiographic findings, 19 differential diagnoses and 104 anatomic locations organized as a hierarchical ta…
Medical image annotation typically requires expert knowledge and hence incurs time-consuming and expensive data annotation costs. To alleviate this burden, we propose a novel learning scenario, Exemplar Learning (EL), to explore automated learning processes for medical image segmentation with a single annotated image example. This innovative learning task is particularly suitable for medical image segmentation, where all categories of organs can be presented in one single image and annotated all at once. To address this challenging EL task, we propose an Exemplar Learning-based Synthesis Net (…
Institute of Imaging and Computer Vision, RWTH Aachen University（亚琛工业大学成像与计算机视觉研究所）
Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotatio…
Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
School of Basic Medical Science, Fudan University（复旦大学基础医学院）；Digital Medical Research Center（数字医学研究中心）；Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention（上海市医学影像计算与计算机辅助介入重点实验室）
Clinical Imaging Research Centre, Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院临床影像研究中心）；Savana Medica（萨瓦纳医疗公司）；Department of Orthopaedic Surgery, National University Hospital, National University Health System（新加坡国立大学卫生系统国立大学医院骨科）；Department of Surgery, Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院外科）；BioIonic Currents Electromagnetic Pulsing Systems Laboratory, BICEPS, National University of Singapore（新加坡国立大学生物离子电流电磁脉冲系统实验室）
Objective: Medical image datasets with pixel-level labels tend to have a limited number of organ or tissue label classes annotated, even when the images have wide anatomical coverage. With supervised learning, multiple classifiers are usually needed given these partially annotated datasets. In this work, we propose a set of strategies to train one single classifier in segmenting all label classes that are heterogeneously annotated across multiple datasets without moving into semi-supervised learning. Methods: Masks were first created from each label image through a process we termed presence m…
