Multi-organ segmentation in medical images is a widely researched task and can save much manual efforts of clinicians in daily routines. Automating the organ segmentation process using deep learning (DL) is a promising solution and state-of-the-art segmentation models are achieving promising accuracy. In this work, We proposed a novel data augmentation strategy for increasing the generalizibility of multi-organ segmentation datasets, namely AnatoMix. By object-level matching and manipulation, our method is able to generate new images with correct anatomy, i.e. organ segmentation mask, exponentially increasing the size of the segmentation dataset. Initial experiments have been done to investigate the segmentation performance influenced by our method on a public CT dataset. Our augmentation method can lead to mean dice of 76.1, compared with 74.8 of the baseline method.
Imperial College London（帝国理工学院）；Universitat Pompeu Fabra（庞培法布拉大学）；Galgo Medical S.L.（Galgo医疗有限公司）；Nara Institute of Science and Technology (NAIST)（奈良先端科学技术大学院大学（NAIST））；Institució Catalana de Recerca i Estudis Avancats (ICREA)（加泰罗尼亚研究与高级研究机构（ICREA））；Children’s National Health System（儿童国家卫生系统）；George Washington University（乔治华盛顿大学）
The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 propo…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Georgia Institute of Technology（佐治亚理工学院）
Thoracic disease detection from chest radiographs using deep learning methods has been an active area of research in the last decade. Most previous methods attempt to focus on the diseased organs of the image by identifying spatial regions responsible for significant contributions to the model's prediction. In contrast, expert radiologists first locate the prominent anatomical structures before determining if those regions are anomalous. Therefore, integrating anatomical knowledge within deep learning models could bring substantial improvement in automatic disease classification. Motivated by …
AG-CRC: Anatomy-Guided Colorectal Cancer Segmentation in CT with Imperfect Anatomical Knowledge
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Digital Medicine Innovation Center（上海数字医学创新中心）；Ruijin Hospital（瑞金医院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）
AbdomenGen: Sequential Volume-Conditioned Diffusion Framework for Abdominal Anatomy Generation
Center for Virtual Imaging Trials, Dept. of Radiology, Duke University School of Medicine（虚拟成像试验中心，放射科，杜克大学医学部）；Dept. of Electrical and Computer Engineering, Pratt School of Engineering, Duke University（电气与计算机工程系，普拉特工程学院，杜克大学）；Dept. of Physics, Trinity School of Arts and Sciences, Duke University（物理系，特里尼蒂艺术与科学学院，杜克大学）
AbdomenGen提出了一种序列体积条件扩散框架，通过Volume Control Scalar实现可控的腹部解剖生成，实现了高几何保真度和多器官解耦调节。
Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院北京协和医院）；Hupan Laboratory（湖畔实验室）；Fuzhou University（福州大学）；Eye & ENT Hospital, Fudan University（复旦大学附属眼耳鼻喉科医院）；Zhongshan Hospital, Fudan University（复旦大学附属中山医院）；Chang Gung Memorial Hospital（长庚纪念医院）；Zhongshan Hospital(Xiamen), Fudan University（复旦大学附属中山医院厦门医院）
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
Vision transformers, with their ability to more efficiently model long-range context, have demonstrated impressive accuracy gains in several computer vision and medical image analysis tasks including segmentation. However, such methods need large labeled datasets for training, which is hard to obtain for medical image analysis. Self-supervised learning (SSL) has demonstrated success in medical image segmentation using convolutional networks. In this work, we developed a \underline{s}elf-distillation learning with \underline{m}asked \underline{i}mage modeling method to perform SSL for vision \u…
seg2med: a bridge from artificial anatomy to multimodal medical images
Computer Assisted Clinical Medicine, Medical Faculty Mannheim, Heidelberg University（计算机辅助临床医学，曼海姆医学院，海德堡大学）；Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg（模式识别实验室，埃尔兰根-纽伦堡弗里德里希-亚历山大大学）；Department of Radiology and Nuclear Medicine, University Medical Center Mannheim（放射学与核医学系，曼海姆大学医学中心）；Mannheim Institute for Intelligent Systems in Medicine, Medical Faculty Mannheim, Heidelberg University（曼海姆智能医学研究所，曼海姆医学院，海德堡大学）；Optical Bioimaging Laboratory, Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore（光学生物成像实验室，生物医学工程系，设计与工程学院，新加坡国立大学）
See More, Change Less: Anatomy-Aware Diffusion for Contrast Enhancement
Johns Hopkins University（约翰霍普金斯大学）；University of Copenhagen（哥本哈根大学）；University of Virginia（弗吉尼亚大学）；University of Bologna（博洛尼亚大学）；Italian Institute of Technology（意大利理工学院）；University of Zurich（苏黎世大学）；ETH AI Center（ETH人工智能中心）；Istanbul Medipol University（伊斯坦布尔梅迪波尔大学）；Jagiellonian University（雅盖隆大学）；NVIDIA（NVIDIA公司）；Johns Hopkins Medicine（约翰霍普金斯医学）；University of California, San Francisco（旧金山大学）
ORACLE-CT: Anatomy-Aware Support Pooling for CT Classification
Center for Virtual Imaging Trials, RAI Labs, Department of Radiology, Duke University（虚拟成像试验中心，RAI实验室，放射学系，杜克大学）；Electrical and Computer Engineering, Pratt School of Engineering, Duke University（电气与计算机工程，工程学院，杜克大学）；Department of Mathematics, Trinity College of Arts & Sciences, Duke University（数学系，艺术与科学学院，杜克大学）；Department of Radiology and Imaging Sciences, University of Arizona College of Medicine（放射学与影像科学系，亚利桑那大学医学院）
CT-GRAPH: Hierarchical Graph Attention Network for Anatomy-Guided CT Report Generation
Institute for AI in Medicine (IKIM), University Hospital Essen (AöR)（人工智能医学研究所（IKIM）、埃森大学医院（AöR））；Department of Nuclear Medicine, University Hospital Essen (AöR)（核医学系、埃森大学医院（AöR））；Department of Physics, TU Dortmund（物理系、多特蒙德技术大学）
Automated classification of human anatomy is an important prerequisite for many computer-aided diagnosis systems. The spatial complexity and variability of anatomy throughout the human body makes classification difficult. "Deep learning" methods such as convolutional networks (ConvNets) outperform other state-of-the-art methods in image classification tasks. In this work, we present a method for organ- or body-part-specific anatomical classification of medical images acquired using computed tomography (CT) with ConvNets. We train a ConvNet, using 4,298 separate axial 2D key-images to learn 5 a…
Self-Supervised Learning for Medical Image Data with Anatomy-Oriented Imaging Planes
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Jarvis Research Center, Tencent YouTu Lab（腾讯优图实验室贾维斯研究中心）
University of Waterloo（滑铁卢大学）；Suez Canal University（苏伊士运河大学）
Quantifying the accuracy of segmentation and manual delineation of organs, tissue types and tumors in medical images is a necessary measurement that suffers from multiple problems. One major shortcoming of all accuracy measures is that they neglect the anatomical significance or relevance of different zones within a given segment. Hence, existing accuracy metrics measure the overlap of a given segment with a ground-truth without any anatomical discrimination inside the segment. For instance, if we understand the rectal wall or urethral sphincter as anatomical zones, then current accuracy measu…
Aligning Fetal Anatomy with Kinematic Tree Log-Euclidean PolyRigid Transforms
Computer Science and Artificial Intelligence Lab, MIT, Cambridge, USA（麻省理工学院计算机科学与人工智能实验室）；California Institute of Technology, Pasadena, USA（加州理工学院）；Boston Children’s Hospital and Harvard Medical School, Boston, USA（哈佛医学院波士顿儿童医院）；The Hebrew University of Jerusalem, Jerusalem, Israel（耶路撒冷希伯来大学）
Training deep learning models on cardiac magnetic resonance imaging (CMR) can be a challenge due to the small amount of expert generated labels and inherent complexity of data source. Self-supervised contrastive learning (SSCL) has recently been shown to boost performance in several medical imaging tasks. However, it is unclear how much the pre-trained representation reflects the primary organ of interest compared to spurious surrounding tissue. In this work, we evaluate the optimal method of incorporating prior knowledge of anatomy into a SSCL training paradigm. Specifically, we evaluate usin…
AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning
Department of Radiology, Northwestern University, Chicago, IL, USA（西北大学放射科）；Data Science Institute, Vanderbilt University, Nashville, TN, USA（范德比尔特大学数据科学研究院）；Department of Orthopedics, The Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二医院骨科）；Second Clinical Medical College, Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二临床医学院）；School of Computer Science, University of Sydney, Sydney, NSW, Australia（悉尼大学计算机科学学院）
School of Psychology, The University of Queensland（昆士兰大学心理学院）；Queensland Brain Institute, The University of Queensland（昆士兰大学昆士兰脑研究所）；Centre for Advanced Imaging, The University of Queensland（昆士兰大学高级成像中心）
Whether it be in a man-made machine or a biological system, form and function are often directly related. In the latter, however, this particular relationship is often unclear due to the intricate nature of biology. Here we developed a geometric deep learning model capable of exploiting the actual structure of the cortex to learn the complex relationship between brain function and anatomy from structural and functional MRI data. Our model was not only able to predict the functional organization of human visual cortex from anatomical properties alone, but it was also able to predict nuanced var…
Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining
Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University（沃森·H·库勒生物医学工程系，佐治亚理工学院和埃默里大学）；Department of Radiation Oncology and Winship Cancer Institute, Emory University（放射肿瘤学系和Winship癌症研究所，埃默里大学）；Department of Electrical and Computer Engineering, Duke University（电气与计算机工程系，杜克大学）；Department of Computer Science and Informatics, Emory University（计算机科学与信息学系，埃默里大学）；Department of Materials Science & Engineering, Nuclear Engineering Program, University of Florida（材料科学与工程系、核工程项目，佛罗里达大学）
ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（生物医学工程学院，生命科学与医学系，中国科学技术大学）；Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE) Lab, YRD-RIGHT, USTC Suzhou Institute for Advanced Research（医学影像、机器人、分析计算与学习（MIRACLE）实验室，YRD-RIGHT，中国科学技术大学苏州研究院）；Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology（江苏省多模态数字孪生技术重点实验室）；Biomedical Basic Research Center (BBRC) of Jiangsu Province（江苏省生物医学基础研究中心）；Department of Radiology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, USTC（放射科，中国科学技术大学第一附属医院，生命科学与医学系，中国科学技术大学）；Anhui IFLYTEK CO., Ltd（安徽科大讯飞股份有限公司）；School of Medicine, Stanford University（医学院，斯坦福大学）；State Key Laboratory of Precision and Intelligent Chemistry, Hefei, Anhui, China（安徽省精密与智能化学重点实验室，合肥，安徽，中国）
Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；The First Affiliated Hospital of College of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Hupan Lab（虎盘实验室）
Statistical shape modeling (SSM) is a valuable and powerful tool to generate a detailed representation of complex anatomy that enables quantitative analysis and the comparison of shapes and their variations. SSM applies mathematics, statistics, and computing to parse the shape into a quantitative representation (such as correspondence points or landmarks) that will help answer various questions about the anatomical variations across the population. Complex anatomical structures have many diverse parts with varying interactions or intricate architecture. For example, the heart is four-chambered…
EXACT: an explainable anomaly-aware vision foundation model for analysis of 3D chest CT
School of Biomedical Engineering, Tsinghua University（清华大学生物医学工程学院）；Department of Radiology, Mianyang Central Hospital（绵阳市中心医院放射科）；Center for Biomedical Imaging Research, Tsinghua University（清华大学生物医学成像研究中心）
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；University of Lübeck（吕贝克大学）
The construction of three-dimensional multi-modal tissue maps provides an opportunity to spur interdisciplinary innovations across temporal and spatial scales through information integration. While the preponderance of effort is allocated to the cellular level and explore the changes in cell interactions and organizations, contextualizing findings within organs and systems is essential to visualize and interpret higher resolution linkage across scales. There is a substantial normal variation of kidney morphometry and appearance across body size, sex, and imaging protocols in abdominal computed…
Shanghai Jiao Tong University（上海交通大学）；University of Sydney（悉尼大学）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院瑞金医院）
Motion estimation is a fundamental step in dynamic medical image processing for the assessment of target organ anatomy and function. However, existing image-based motion estimation methods, which optimize the motion field by evaluating the local image similarity, are prone to produce implausible estimation, especially in the presence of large motion. In this study, we provide a novel motion estimation framework of Dense-Sparse-Dense (DSD), which comprises two stages. In the first stage, we process the raw dense image to extract sparse landmarks to represent the target organ anatomical topology…
GLeVE: Graph-Guided Lesion Grounding with Proposal Verification in 3D CT
Zhejiang Key Laboratory of Space Information Sensing and Transmission（浙江空间信息感知与传输重点实验室）；Hangzhou Dianzi University（杭州电子科技大学）；Zhejiang University（浙江大学）；Tsinghua University（清华大学）；Children's Hospital, Zhejiang University School of Medicine（浙江大学医学院附属儿童医院）
Two-View Topogram-Based Anatomy-Guided CT Reconstruction for Prospective Risk Minimization
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡大学模式识别实验室）；Division of X-Ray Imaging and Computed Tomography, German Cancer Research Center (DKFZ)（德国癌症研究中心X射线成像与计算机断层扫描部门）；Department of Physics and Astronomy, Ruprecht-Karls-University Heidelberg（海德堡大学物理与天文学系）；Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡大学埃尔朗根大学医院放射肿瘤科）；Department of Radiology and Nuclear Medicine, Klinikum Nürnberg, Paracelsus Medical University（帕拉塞尔苏斯医科大学纽伦堡医院放射与核医学科）
Anatomy-constrained modelling of image-derived input functions in dynamic PET using multi-organ segmentation
Department of diagnostic and interventional Radiology and Neuroradiology, University Hospital Augsburg（诊断与介入放射学及神经放射学系，奥格斯堡大学医院）；Computer-Aided Medical Procedures and Augmented Reality, Technical University of Munich（计算机辅助医疗程序与增强现实，慕尼黑技术大学）；Digital Medicine, University Hospital Augsburg（数字医学，奥格斯堡大学医院）；Center of Advanced Analytics and Predictive Sciences, University of Augsburg（高级分析与预测科学中心，奥格斯堡大学）
Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Inpainting has recently been proposed as a successful deep learning technique for unsupervised medical image model discovery. The masks used for inpainting are generally independent of the dataset and are not tailored to perform on different given classes of anatomy. In this work, we introduce a method for generating shape-aware masks for inpainting, which aims at learning the statistical shape prior. We hypothesize that although the variation of masks improves the generalizability of inpainting models, the shape of the masks should follow the topology of the organs of interest. Hence, we prop…
A 3D deep learning model (OARnet) is developed and used to delineate 28 H&N OARs on CT images. OARnet utilizes a densely connected network to detect the OAR bounding-box, then delineates the OAR within the box. It reuses information from any layer to subsequent layers and uses skip connections to combine information from different dense block levels to progressively improve delineation accuracy. Training uses up to 28 expert manual delineated (MD) OARs from 165 CTs. Dice similarity coefficient (DSC) and the 95th percentile Hausdorff distance (HD95) with respect to MD is assessed for 70 other C…
Vision Transformers (ViT)s have shown great performance in self-supervised learning of global and local representations that can be transferred to downstream applications. Inspired by these results, we introduce a novel self-supervised learning framework with tailored proxy tasks for medical image analysis. Specifically, we propose: (i) a new 3D transformer-based model, dubbed Swin UNEt TRansformers (Swin UNETR), with a hierarchical encoder for self-supervised pre-training; (ii) tailored proxy tasks for learning the underlying pattern of human anatomy. We demonstrate successful pre-training of…
Microsoft Research（微软研究院）；TuSimple（图森未来）；Baidu Research（百度研究院）；Tencent Medical AI Labs（腾讯医疗AI实验室）；PingAn Tech（平安科技）
Medical imaging is an essential tool in many areas of medical applications, used for both diagnosis and treatment. However, reading medical images and making diagnosis or treatment recommendations require specially trained medical specialists. The current practice of reading medical images is labor-intensive, time-consuming, costly, and error-prone. It would be more desirable to have a computer-aided system that can automatically make diagnosis and treatment recommendations. Recent advances in deep learning enable us to rethink the ways of clinician diagnosis based on medical images. In this t…
PAII Inc.（PAII公司）；Stony Brook University（石溪大学）；Harvard University（哈佛大学）；Ping An Technology（平安科技）；Changhai Hospital（长海医院）
The pancreatic disease taxonomy includes ten types of masses (tumors or cysts)[20,8]. Previous work focuses on developing segmentation or classification methods only for certain mass types. Differential diagnosis of all mass types is clinically highly desirable [20] but has not been investigated using an automated image understanding approach. We exploit the feasibility to distinguish pancreatic ductal adenocarcinoma (PDAC) from the nine other nonPDAC masses using multi-phase CT imaging. Both image appearance and the 3D organ-mass geometry relationship are critical. We propose a holistic segme…
Imperial College London（帝国理工学院）；UK Dementia Research Institute（英国痴呆症研究所）
Segmentation of image sequences is an important task in medical image analysis, which enables clinicians to assess the anatomy and function of moving organs. However, direct application of a segmentation algorithm to each time frame of a sequence may ignore the temporal continuity inherent in the sequence. In this work, we propose an image sequence segmentation algorithm by combining a fully convolutional network with a recurrent neural network, which incorporates both spatial and temporal information into the segmentation task. A key challenge in training this network is that the available ma…
