Comprehensive Validation of Automated Whole Body Skeletal Muscle, Adipose Tissue, and Bone Segmentation from 3D CT images for Body Composition Analysis: Towards Extended Body Composition
Simon Fraser University（西蒙弗雷泽大学）；Voronoi Health Analytics Incorporated（沃罗诺伊健康分析公司）
Da Ma, Vincent Chow, Karteek Popuri, Mirza Faisal Beg
备注：
This paper is based on concepts presented at the NIH Body Composition and Cancer Outcomes Research Webinar Series on December 17th, 2020 by Mirza Faisal Beg titled "Automating Body Composition from Routinely Acquired CT images - towards 3D measurements". The talk is archived [here](https://epi.grants.cancer.gov/events/body-composition/#past)
The latest advances in computer-assisted precision medicine are making it feasible to move from population-wide models that are useful to discover aggregate patterns that hold for group-based analysis to patient-specific models that can drive patient-specific decisions with regard to treatment choices, and predictions of outcomes of treatment. Body Composition is recognized as an important driver and risk factor for a wide variety of diseases, as well as a predictor of individual patient-specific clinical outcomes to treatment choices or surgical interventions. 3D CT images are routinely acquired in the oncological worklows and deliver accurate rendering of internal anatomy and therefore can be used opportunistically to assess the amount of skeletal muscle and adipose tissue compartments. Powerful tools of artificial intelligence such as deep learning are making it feasible now to segment the entire 3D image and generate accurate measurements of all internal anatomy. These will enable the overcoming of the severe bottleneck that existed previously, namely, the need for manual segmentation, which was prohibitive to scale to the hundreds of 2D axial slices that made up a 3D volumetric image. Automated tools such as presented here will now enable harvesting whole-body measurements from 3D CT or MRI images, leading to a new era of discovery of the drivers of various diseases based on individual tissue, organ volume, shape, and functional status. These measurements were hitherto unavailable thereby limiting the field to a very small and limited subset. These discoveries and the potential to perform individual image segmentation with high speed and accuracy are likely to lead to the incorporation of these 3D measures into individual specific treatment planning models related to nutrition, aging, chemotoxicity, surgery and survival after the onset of a major disease such as cancer.
Concordia Institute for Information Systems Engineering（康考迪亚大学信息系统工程研究所）；Concordia University（康考迪亚大学）
Bone age is an important measure for assessing the skeletal and biological maturity of children. Delayed or increased bone age is a serious concern for pediatricians, and needs to be accurately assessed in a bid to determine whether bone maturity is occurring at a rate consistent with chronological age. In this paper, we introduce a unified deep learning framework for bone age assessment using instance segmentation and ridge regression. The proposed approach consists of two integrated stages. In the first stage, we employ an image annotation and segmentation model to annotate and segment the h…
BoneAgeTW2: Automated Skeletal Maturation Assessment via the Tanner-Whitehouse 2 Method, Deep Learning, and Clinical Report Generation with Distribution Curves
Universityhospital Tübingen（图宾根大学医院）；University Hospital of Padua（帕多瓦大学医院）；Hopital Universitaire Henri Mondor（亨利·蒙多大学医院）；University of Genoa（热那亚大学）；University of Tübingen（图宾根大学）
To meet the current need for skeletal tumor-load estimation in prostate cancer (mCRPC), we developed a novel approach, based on adaptive bone segmentation. In this study, we compared the program output with existing estimates and with the radiological outcome. Seventy-six whole-body 99mTc-DPD-SPECT/CT from mCRPC patients were analyzed. The software identified the whole skeletal volume (SVol) and classified it voxels metastases (MVol) or normal bone (BVol). SVol was compared with the estimation of a commercial software. MVol was compared with manual assessment and with PSA-level. Counts/voxel w…
Bone age assessment is an important clinical trial to measure skeletal child maturity and diagnose of growth disorders. Conventional approaches such as the Tanner-Whitehouse (TW) and Greulich and Pyle (GP) may not perform well due to their large inter-observer and intra-observer variations. In this paper, we propose a finger joint localization strategy to filter out most non-informative parts of images. When combining with the conventional full image-based deep network, we observe a much-improved performance. % Our approach utilizes full hand and specific joints images for skeletal maturity pr…
Department of Computing, Imperial College London（伦敦帝国理工学院计算系）
Most realtime human pose estimation approaches are based on detecting joint positions. Using the detected joint positions, the yaw and pitch of the limbs can be computed. However, the roll along the limb, which is critical for application such as sports analysis and computer animation, cannot be computed as this axis of rotation remains unobserved. In this paper we therefore introduce orientation keypoints, a novel approach for estimating the full position and rotation of skeletal joints, using only single-frame RGB images. Inspired by how motion-capture systems use a set of point markers to e…
Lyft Inc.（Lyft公司）；Neuromation OU（Neuromation公司）；University of Michigan（密歇根大学）；Massachusetts Institute of Technology（麻省理工学院）
Skeletal bone age assessment is a common clinical practice to diagnose endocrine and metabolic disorders in child development. In this paper, we describe a fully automated deep learning approach to the problem of bone age assessment using data from Pediatric Bone Age Challenge organized by RSNA 2017. The dataset for this competition is consisted of 12.6k radiological images of left hand labeled by the bone age and sex of patients. Our approach utilizes several deep learning architectures: U-Net, ResNet-50, and custom VGG-style neural networks trained end-to-end. We use images of whole hands as…
Machine Learning-based fast and quantitative automated screening plays a key role in analyzing human bones on Computed Tomography (CT) scans. However, despite the requirement in drug safety assessment, such research is rare on animal fetus micro-CT scans due to its laborious data collection and annotation. Therefore, we propose various bone feature engineering techniques to thoroughly automate the skeletal localization/labeling/abnormality detection of rat fetuses on whole-body micro-CT scans with minimum effort. Despite limited training data of 49 fetuses, in skeletal labeling and abnormality…
Body tissue composition is a long-known biomarker with high diagnostic and prognostic value in cardiovascular, oncological and orthopaedic diseases, but also in rehabilitation medicine or drug dosage. In this study, the aim was to develop a fully automated, reproducible and quantitative 3D volumetry of body tissue composition from standard CT examinations of the abdomen in order to be able to offer such valuable biomarkers as part of routine clinical imaging. Therefore an in-house dataset of 40 CTs for training and 10 CTs for testing were fully annotated on every fifth axial slice with five di…
University of Pittsburgh School of Medicine（匹兹堡大学医学院）；VA Pittsburgh Healthcare System（匹兹堡退伍军人医疗系统）
Purpose: To develop and validate a computer tool for automatic and simultaneous segmentation of body composition depicted on computed tomography (CT) scans for the following tissues: visceral adipose (VAT), subcutaneous adipose (SAT), intermuscular adipose (IMAT), skeletal muscle (SM), and bone. Approach: A cohort of 100 CT scans acquired from The Cancer Imaging Archive (TCIA) was used - 50 whole-body positron emission tomography (PET)-CTs, 25 chest, and 25 abdominal. Five different body compositions were manually annotated (VAT, SAT, IMAT, SM, and bone). A training-while-annotating strategy w…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；IMATI-CNR（意大利国家研究委员会信息技术与应用数学研究所）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立理工学院）
We address the problem of inferring the anatomic skeleton of a person, in an arbitrary pose, from the 3D surface of the body; i.e. we predict the inside (bones) from the outside (skin). This has many applications in medicine and biomechanics. Existing state-of-the-art biomechanical skeletons are detailed but do not easily generalize to new subjects. Additionally, computer vision and graphics methods that predict skeletons are typically heuristic, not learned from data, do not leverage the full 3D body surface, and are not validated against ground truth. To our knowledge, our system, called OSS…
Stanford University（斯坦福大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Mayo Clinic（梅奥诊所）；Wake Forest University（维克森林大学）
Computed tomography (CT) is routinely used in clinical practice to evaluate a wide variety of medical conditions. While CT scans provide diagnoses, they also offer the ability to extract quantitative body composition metrics to analyze tissue volume and quality. Extracting quantitative body composition measures manually from CT scans is a cumbersome and time-consuming task. Proprietary software has been developed recently to automate this process, but the closed-source nature impedes widespread use. There is a growing need for fully automated body composition software that is more accessible a…
Automated Muscle and Fat Segmentation in Computed Tomography for Comprehensive Body Composition Analysis
Department of Electrical and Computer Engineering, Duke University（杜克大学电气与计算机工程系）；Department of Radiology, Duke University（杜克大学放射学系）；Department of Surgery Duke University School of Medicine（杜克大学医学学院外科系）；Department of Biostatistics & Bioinformatics, Duke University（杜克大学生物统计学与生物信息学系）；Department of Computer Science, Duke University（杜克大学计算机科学系）
Estimation of 3D human pose from monocular image has gained considerable attention, as a key step to several human-centric applications. However, generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable, as these models often perform unsatisfactorily on unseen in-the-wild environments. Though weakly-supervised models have been proposed to address this shortcoming, performance of such models relies on availability of paired supervision on some related tasks, such as 2D pose or multi-view image pairs. In contrast, we prop…
Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization
South China University of Technology（南方科技大学）；Hong Kong University of Science and Technology（香港科技大学）；National University of Singapore（国立新加坡大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；University of Science and Technology of China（中国科学技术大学）；Beijing Normal University（北京师范大学）；Tencent Hunyuan（腾讯文盈）
University of Delaware（特拉华大学）；Amazon Web Services（亚马逊云科技）；University of Rochester（罗切斯特大学）
Objective and Impact Statement. We adopt a deep learning model for bone osteolysis prediction on computed tomography (CT) images of murine breast cancer bone metastases. Given the bone CT scans at previous time steps, the model incorporates the bone-cancer interactions learned from the sequential images and generates future CT images. Its ability of predicting the development of bone lesions in cancer-invading bones can assist in assessing the risk of impending fractures and choosing proper treatments in breast cancer bone metastasis. Introduction. Breast cancer often metastasizes to bone, cau…
National Institute of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））；CNRS-AIST JRL (Joint Robotics Laboratory), UMI3218/RL（法国国家科学研究中心-国立先进工业科学技术研究所联合机器人实验室（UMI3218/RL））
In this paper, we present Skeleton Transformer Networks (SkeletonNet), an end-to-end framework that can predict not only 3D joint positions but also 3D angular pose (bone rotations) of a human skeleton from a single color image. This in turn allows us to generate skinned mesh animations. Here, we propose a two-step regression approach. The first step regresses bone rotations in order to obtain an initial solution by considering skeleton structure. The second step performs refinement based on heatmap regressor using a 3D pose representation called cross heatmap which stacks heatmaps of xy and z…
LegSegNet: A Public Deep Learning System for Lower Extremity CT Tissue Segmentation and Quantification
Department of Electrical and Computer Engineering, Duke University（杜克大学电气与计算机工程系）；Department of Biostatistics & Bioinformatics, Duke University（杜克大学生物统计与生物信息学系）；Department of Radiology, Duke University（杜克大学放射学系）；Department of Computer Science, Duke University（杜克大学计算机科学系）；Department of Surgery, Duke University（杜克大学外科系）
Transformer-Based Multi-Region Segmentation and Radiomic Analysis of HR-pQCT Imaging for Osteoporosis Classification
Weldon School of Biomedical Engineering, Purdue University（普渡大学生物医学工程学院）；Division of Bone and Mineral Diseases, Washington University Medicine（华盛顿大学医学院骨科与矿物质疾病部）；Department of Anatomy, Cell Biology & Physiology, Indiana University School of Medicine（印第安纳大学医学院解剖学、细胞生物学与生理学部）；Department of Physical Therapy, School of Health & Human Sciences, Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校物理治疗系）
In addition to body weight and Body Mass Index (BMI), body composition is an essential data point that allows people to understand their overall health and body fitness. However, body composition is largely made up of muscle, fat, bones, and water, which makes estimation not as easy and straightforward as measuring body weight. In this paper, we introduce a multimodal multi-task deep neural network to estimate body fat percentage and skeletal muscle mass by analyzing facial images in addition to a person's height, gender, age, and weight information. Using a dataset representative of demograph…
Islamic University of Technology（伊斯兰科技大学）；University of South Florida（南佛罗里达大学）
To guide surgical and medical treatment X-ray images have been used by physicians in every modern healthcare organization and hospitals. Doctor's evaluation process and disease identification in the area of skeletal system can be performed in a faster and efficient way with the help of X-ray imaging technique as they can depict bone structure painlessly. This paper presents an efficient contrast enhancement technique using morphological operators which will help to visualize important bone segments and soft tissues more clearly. Top-hat and Bottom-hat transform are utilized to enhance the imag…
UniFLM: United Segmentation and Measurement on Fetal Limb Ultrasonic Image
Academy of Advanced Interdisciplinary Studies, Wuhan University（武汉大学高等研究院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Guangzhou Women and Children’s Medical Center（广州市妇女儿童医疗中心）；Institute of Artificial Intelligence, School of Computer Science, Wuhan University（武汉大学计算机学院人工智能研究所）
IMT Atlantique（大西洋国立高等矿业电信学院）；LaTIM UMR 1101, Inserm（法国国家健康与医学研究院 1101 联合研究单位（医学影像治疗实验室））
This paper proposes an automatic method for scapula bone segmentation from Magnetic Resonance (MR) images using deep learning. The purpose of this work is to incorporate anatomical priors into a conditional adversarial framework, given a limited amount of heterogeneous annotated images. Our approach encourages the segmentation model to follow the global anatomical properties of the underlying anatomy through a learnt non-linear shape representation while the adversarial contribution refines the model by promoting realistic delineations. These contributions are evaluated on a dataset of 15 pedi…
University of California, San Diego（加州大学圣地亚哥分校）；Qilu University of Technology（齐鲁工业大学）
Acupuncture is a technique in which practitioners stimulate specific points on the body. Those points, called acupuncture points (or acupoints), anatomically define areas on the skin relative to specific landmarks on the body. However, mapping the acupoints to individuals could be challenging for inexperienced acupuncturists. In this project, we proposed a system to localize and visualize facial acupoints for individuals in an augmented reality (AR) context. This system combines a face alignment model and a hair segmentation model to provide dense reference points for acupoints localization in…
