Imposing key anatomical features, such as the number of organs, their shapes and relative positions, is crucial for building a robust multi-organ segmentation model. Current attempts to incorporate anatomical features include broadening the effective receptive field (ERF) size with data-intensive modules, or introducing anatomical constraints that scales poorly to multi-organ segmentation. We introduce a novel architecture called the Anatomy-Informed Cascaded Segmentation Network (AIC-Net). AIC-Net incorporates a learnable input termed "Anatomical Prior", which can be adapted to patient-specific anatomy using a differentiable spatial deformation. The deformed prior later guides decoder layers towards more anatomy-informed predictions. We repeat this process at a local patch level to enhance the representation of intricate objects, resulting in a cascaded network structure. AIC-Net is a general method that enhances any existing segmentation models to be more anatomy-aware. We have validated the performance of AIC-Net, with various backbones, on two multi-organ segmentation tasks: abdominal organs and vertebrae. For each respective task, our benchmarks demonstrate improved dice score and Hausdorff distance.
We hypothesize that probabilistic voxel-level classification of anatomy and malignancy in prostate MRI, although typically posed as near-identical segmentation tasks via U-Nets, require different loss functions for optimal performance due to inherent differences in their clinical objectives. We investigate distribution, region and boundary-based loss functions for both tasks across 200 patient exams from the publicly-available ProstateX dataset. For evaluation, we conduct a thorough comparative analysis of model predictions and calibration, measured with respect to multi-class volume segmentat…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Georgia Institute of Technology（佐治亚理工学院）
Thoracic disease detection from chest radiographs using deep learning methods has been an active area of research in the last decade. Most previous methods attempt to focus on the diseased organs of the image by identifying spatial regions responsible for significant contributions to the model's prediction. In contrast, expert radiologists first locate the prominent anatomical structures before determining if those regions are anomalous. Therefore, integrating anatomical knowledge within deep learning models could bring substantial improvement in automatic disease classification. Motivated by …
Towards Unifying Anatomy Segmentation: Automated Generation of a Full-body CT Dataset via Knowledge Aggregation and Anatomical Guidelines
Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院人机交互与机器人学研究所）；Institute for AI in Medicine, University Hospital Essen（埃森大学医院医学人工智能研究所）；Steinbuch Center for Computing, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT）施泰因布赫计算中心）；Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ）放射肿瘤学医学物理科）；Heidelberg Institute of Radiation Oncology (HIRO)（海德堡放射肿瘤学研究所（HIRO））；National Center for Radiation Research in Oncology (NCRO)（国家肿瘤放射研究中心（NCRO））；Helmholtz Information and Data Science School for Health (HIDSS4Health)（亥姆霍兹健康信息与数据科学学院（HIDSS4Health））；Department of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen（埃森大学医院诊断与介入放射学及神经放射学系）
Anatomy-Informed Deep Learning for Abdominal Aortic Aneurysm Segmentation
Faculty of Mathematics, Informatics and Technology, University of Applied Sciences Koblenz（科布伦茨应用科学大学数学、信息学与技术学院）；Department of Radiology, Herz-Jesu-Krankenhaus（赫茨-耶苏医院放射科）
Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与成像研究所）；School of Computing, University of Utah（犹他大学计算学院）
Statistical shape modeling is an essential tool for the quantitative analysis of anatomical populations. Point distribution models (PDMs) represent the anatomical surface via a dense set of correspondences, an intuitive and easy-to-use shape representation for subsequent applications. These correspondences are exhibited in two coordinate spaces: the local coordinates describing the geometrical features of each individual anatomical surface and the world coordinates representing the population-level statistical shape information after removing global alignment differences across samples in the …
GRASPing Anatomy to Improve Pathology Segmentation
Karlsruhe Institute of Technology, Karlsruhe, Germany（卡尔斯鲁厄理工学院）；Helmholtz Information and Data Science School for Health, Germany（海德堡信息与数据科学健康学校）；Department of Nuclear Medicine, University of Duisburg-Essen（杜伊斯堡-埃森大学核医学系）；German Cancer Consortium (DKTK)- University Hospital Essen, Essen, Germany（德国癌症研究中心（DKTK）-埃森大学医院）
ASCON: Anatomy-aware Supervised Contrastive Learning Framework for Low-dose CT Denoising
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）
Creating a large-scale dataset of abnormality annotation on medical images is a labor-intensive and costly task. Leveraging weak supervision from readily available data such as radiology reports can compensate lack of large-scale data for anomaly detection methods. However, most of the current methods only use image-level pathological observations, failing to utilize the relevant anatomy mentions in reports. Furthermore, Natural Language Processing (NLP)-mined weak labels are noisy due to label sparsity and linguistic ambiguity. We propose an Anatomy-Guided chest X-ray Network (AGXNet) to addr…
AG-CRC: Anatomy-Guided Colorectal Cancer Segmentation in CT with Imperfect Anatomical Knowledge
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Digital Medicine Innovation Center（上海数字医学创新中心）；Ruijin Hospital（瑞金医院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）
University of Oxford（牛津大学）；Tel-Aviv University（特拉维夫大学）
Self-supervised contrastive representation learning offers the advantage of learning meaningful visual representations from unlabeled medical datasets for transfer learning. However, applying current contrastive learning approaches to medical data without considering its domain-specific anatomical characteristics may lead to visual representations that are inconsistent in appearance and semantics. In this paper, we propose to improve visual representations of medical images via anatomy-aware contrastive learning (AWCL), which incorporates anatomy information to augment the positive/negative pa…
University of South Florida（南佛罗里达大学）；Northern Illinois University（北伊利诺伊大学）
We design a framework based on Mask Region-based Convolutional Neural Network (Mask R-CNN) to automatically detect and separately extract anatomical components of mosquitoes - thorax, wings, abdomen and legs from images. Our training dataset consisted of 1500 smartphone images of nine mosquito species trapped in Florida. In the proposed technique, the first step is to detect anatomical components within a mosquito image. Then, we localize and classify the extracted anatomical components, while simultaneously adding a branch in the neural network architecture to segment pixels containing only t…
CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs
Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University（斯坦福大学医学与影像人工智能中心）；Department of Radiology, Stanford University（斯坦福大学放射学系）
We present GAAF, a Generalised Automatic Anatomy Finder, for the identification of generic anatomical locations in 3D CT scans. GAAF is an end-to-end pipeline, with dedicated modules for data pre-processing, model training, and inference. At it's core, GAAF uses a custom a localisation convolutional neural network (CNN). The CNN model is small, lightweight and can be adjusted to suit the particular application. The GAAF framework has so far been tested in the head and neck, and is able to find anatomical locations such as the centre-of-mass of the brainstem. GAAF was evaluated in an open-acces…
Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）
Fetal heart motion is an important diagnostic indicator for structural detection and functional assessment of congenital heart disease. We propose an approach towards integrating deep convolutional and recurrent architectures that utilize localized spatial and temporal features of different anatomical substructures within a global spatiotemporal context for interpretation of fetal echocardiography videos. We formulate our task as a cardiac structure localization problem with convolutional architectures for aggregating global spatial context and detecting anatomical structures on spatial region…
We present a semi-parametric generative model for predicting anatomy of a patient in subsequent scans following a single baseline image. Such predictive modeling promises to facilitate novel analyses in both voxel-level studies and longitudinal biomarker evaluation. We capture anatomical change through a combination of population-wide regression and a non-parametric model of the subject's health based on individual genetic and clinical indicators. In contrast to classical correlation and longitudinal analysis, we focus on predicting new observations from a single subject observation. We demons…
University of Waterloo（滑铁卢大学）；Suez Canal University（苏伊士运河大学）
Quantifying the accuracy of segmentation and manual delineation of organs, tissue types and tumors in medical images is a necessary measurement that suffers from multiple problems. One major shortcoming of all accuracy measures is that they neglect the anatomical significance or relevance of different zones within a given segment. Hence, existing accuracy metrics measure the overlap of a given segment with a ground-truth without any anatomical discrimination inside the segment. For instance, if we understand the rectal wall or urethral sphincter as anatomical zones, then current accuracy measu…
Nanjing University of Information Science & Technology（南京信息工程大学）
Cerebrovascular accident, or commonly known as stroke, is an acute disease with extreme impact on patients and healthcare systems and is the second largest cause of death worldwide. Fast and precise stroke lesion detection and location is an extreme important process with regards to stroke diagnosis, treatment, and prognosis. Except from the manual segmentation approach, machine learning based segmentation methods are the most promising ones when considering efficiency and accuracy, and convolutional neural network based models are the first of its kind. However, most of these neural network m…
Virginia Tech（弗吉尼亚理工大学）；IBM Research（IBM研究院）；Institute for Medical Engineering and Science, Massachusetts Institute of Technology（麻省理工学院医学工程与科学研究所）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）；University of British Columbia（不列颠哥伦比亚大学）
Despite the progress in utilizing deep learning to automate chest radiograph interpretation and disease diagnosis tasks, change between sequential Chest X-rays (CXRs) has received limited attention. Monitoring the progression of pathologies that are visualized through chest imaging poses several challenges in anatomical motion estimation and image registration, i.e., spatially aligning the two images and modeling temporal dynamics in change detection. In this work, we propose CheXRelNet, a neural model that can track longitudinal pathology change relations between two CXRs. CheXRelNet incorpor…
Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays
Ewha Womans University（梨花女子大学）；Ewha Medical Artificial Intelligence Research Institute（梨花女子大学医学人工智能研究所）；REMEDI Inc.（瑞美迪公司）；Ewha Womans University Seoul Hospital（梨花女子大学首尔医院）
Radiologists usually observe anatomical regions of chest X-ray images as well as the overall image before making a decision. However, most existing deep learning models only look at the entire X-ray image for classification, failing to utilize important anatomical information. In this paper, we propose a novel multi-label chest X-ray classification model that accurately classifies the image finding and also localizes the findings to their correct anatomical regions. Specifically, our model consists of two modules, the detection module and the anatomical dependency module. The latter utilizes g…
seg2med: a bridge from artificial anatomy to multimodal medical images
Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University（计算机辅助临床医学，曼海姆医学院，海德堡大学）；Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg（模式识别实验室，埃尔兰根-纽伦堡弗里德里希-亚历山大大学）；Department of Radiology and Nuclear Medicine, University Medical Center Mannheim（放射学与核医学系，曼海姆大学医学中心）；Mannheim Institute for Intelligent Systems in Medicine, Medical Faculty Mannheim, Heidelberg University（曼海姆智能医学研究所，曼海姆医学院，海德堡大学）；Optical Bioimaging Laboratory, Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore（光学生物成像实验室，生物医学工程系，设计与工程学院，新加坡国立大学）
AFiRe: Anatomy-Driven Self-Supervised Learning for Fine-Grained Representation in Radiographic Images
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Shanghai Eye Diseases Prevention and Treatment Center, Shanghai Eye Hospital（上海市眼病防治中心、上海眼科医院）；School of Communication and Electronic Engineering, East China Normal University（华东师范大学通信与电子工程学院）
MambaControl: Anatomy Graph-Enhanced Mamba ControlNet with Fourier Refinement for Diffusion-Based Disease Trajectory Prediction
Faculty of Applied Sciences, Macao Polytechnic University（应用科学学院，澳门理工学院）；Department of Electrical Engineering, Zhejiang University（电气工程学院，浙江大学）；Department of Computer Science, The University of Adelaide（计算机科学学院，阿德莱德大学）；Department of Computer Science, University of Birmingham（计算机科学学院，伯明翰大学）
Two-View Topogram-Based Anatomy-Guided CT Reconstruction for Prospective Risk Minimization
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡大学模式识别实验室）；Division of X-Ray Imaging and Computed Tomography, German Cancer Research Center (DKFZ)（德国癌症研究中心X射线成像与计算机断层扫描部门）；Department of Physics and Astronomy, Ruprecht-Karls-University Heidelberg（海德堡大学物理与天文学系）；Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡大学埃尔朗根大学医院放射肿瘤科）；Department of Radiology and Nuclear Medicine, Klinikum Nürnberg, Paracelsus Medical University（帕拉塞尔苏斯医科大学纽伦堡医院放射与核医学科）
SAMe: A Semantic Anatomy Mapping Engine for Robotic Ultrasound
School of Computer Science, Wuhan University（武汉大学计算机学院）；Hubei Center for Applied Mathematics, Wuhan University（湖北应用数学中心，武汉大学）；Department of Ultrasound, The Central Hospital of Wuhan（武汉市中心医院超声科）；Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology（同济医院，同济医学院，华中科技大学医学影像科）；Department of Ultrasound in Medicine, Renmin Hospital of Wuhan University（武汉大学仁医医院医学超声科）；University Hospital, Johann-Wolfgang-Goethe University Frankfurt am Main（法兰克福歌德大学医学院大学医院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
FoundDiff: Foundational Diffusion Model for Generalizable Low-Dose CT Denoising
Institute of Science and Technology for Brain-inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University（脑启发智能科学技术研究所和脑科学前沿中心，复旦大学）；Shanghai Key Lab of Intelligent Information Processing, School of Computer Science, Fudan University（智能信息处理上海市重点实验室，复旦大学计算机科学学院）；School of Cyber Science and Engineering, Sichuan University（四川大学网络科学与工程学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）
Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRI
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（马萨诸塞州总医院阿西诺拉·A·马丁诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（马萨诸塞州总医院）
University of Pittsburgh（匹兹堡大学）；Boston University（波士顿大学）
Large-scale volumetric medical images with annotation are rare, costly, and time prohibitive to acquire. Self-supervised learning (SSL) offers a promising pre-training and feature extraction solution for many downstream tasks, as it only uses unlabeled data. Recently, SSL methods based on instance discrimination have gained popularity in the medical imaging domain. However, SSL pre-trained encoders may use many clues in the image to discriminate an instance that are not necessarily disease-related. Moreover, pathological patterns are often subtle and heterogeneous, requiring the ability of the…
F2PASeg: Feature Fusion for Pituitary Anatomy Segmentation in Endoscopic Surgery
1Centre for Artificial Intelligence；Robotics, Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences 2The University of Hong Kong 3Peking Union Medical College Hospital, Chinese Academy of Medical Science
MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images
Boston University（波士顿大学）；Stanford University（斯坦福大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）；University of Pittsburgh（匹兹堡大学）
