Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
Center for Informatics Science, School of Information Technology and Computer Science, Nile University（信息科学中心，信息科技与计算机科学学院，尼罗大学）；Graduate School of Information Science, University of Hyogo（信息科学研究生院，兵库大学）；Advanced Medical Engineering Research Institute, University of Hyogo（先进医学工程研究所，兵库大学）
Ahmed Kabil, Ghada Khoriba, Mina Yousef, Essam A. Rashed
Medical Image Segmentation (MIS) stands as a cornerstone in medical image analysis, playing a pivotal role in precise diagnostics, treatment planning, and monitoring of various medical conditions. This paper presents a comprehensive and systematic survey of MIS methodologies, bridging the gap between traditional image processing techniques and modern deep learning approaches. The survey encompasses thresholding, edge detection, region-based segmentation, clustering algorithms, and model-based techniques while also delving into state-of-the-art deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), and the widely adopted U-Net and its variants. Moreover, integrating attention mechanisms, semi-supervised learning, generative adversarial networks (GANs), and Transformer-based models is thoroughly explored. In addition to covering established methods, this survey highlights emerging trends, including hybrid architectures, cross-modality learning, federated and distributed learning frameworks, and active learning strategies, which aim to address challenges such as limited labeled datasets, computational complexity, and model generalizability across diverse imaging modalities. Furthermore, a specialized case study on lumbar spine segmentation is presented, offering insights into the challenges and advancements in this relatively underexplored anatomical region. Despite significant progress in the field, critical challenges persist, including dataset bias, domain adaptation, interpretability of deep learning models, and integration into real-world clinical workflows.
This paper addresses the challenge of grading visual features in lumbar spine MRI using Deep Learning. Such a method is essential for the automatic quantification of structural changes in the spine, which is valuable for understanding low back pain. Multiple recent studies investigated different architecture designs, and the most recent success has been attributed to the use of transformer architectures. In this work, we argue that with a well-tuned three-stage pipeline comprising semantic segmentation, localization, and classification, convolutional networks outperform the state-of-the-art ap…
The aim of this study is to investigate the segmentation accuracies of different segmentation networks trained on 730 manually annotated lateral lumbar spine X-rays. Instance segmentation networks were compared to semantic segmentation networks. The study cohort comprised diseased spines and postoperative images with metallic implants. The average mean accuracy and mean intersection over union (IoU) was up to 3 percent better for the best performing instance segmentation model, the average pixel accuracy and weighted IoU were slightly better for the best performing semantic segmentation model.…
Panoptic Segmentation and Labelling of Lumbar Spine Vertebrae using Modified Attention Unet
A. K. Choudhury School of Information Technology, University of Calcutta（加尔各答大学A. K.乔杜里信息技术学院）；Rammohan College, University of Calcutta（加尔各答大学拉莫汉学院）；Indian Statistical Institute（印度统计研究所）
本研究提出基于改进注意力 U-Net 架构和新型掩码逻辑的方法，用于 3D 腰椎 MRI 数据的椎骨全景分割与标注，实现 99.5% 的高准确率，显著推进该领域最先进技术。
Fundaciò per al Foment de la Investigaciò Sanitària i Biomèdica (FISABIO)（瓦伦西亚卫生与生物医学研究促进基金会（FISABIO））；Hospital Arnau de Vilanova（阿尔瑙·德·比拉诺瓦医院）；Universitat Politècnica de València（瓦伦西亚理工大学）；Regional ministry of Universal Health and Public Health in Valencia（瓦伦西亚全民健康与公共卫生部）
One of the major difficulties in medical image segmentation is the high variability of these images, which is caused by their origin (multi-centre), the acquisition protocols (multi-parametric), as well as the variability of human anatomy, the severity of the illness, the effect of age and gender, among others. The problem addressed in this work is the automatic semantic segmentation of lumbar spine Magnetic Resonance images using convolutional neural networks. The purpose is to assign a class label to each pixel of an image. Classes were defined by radiologists and correspond to different str…
Insight Centre for Data Analytics, Dublin City University（都柏林城市大学数据洞察中心）；Sports Medicine Department, Sports Surgery Clinic（运动外科诊所运动医学部）；School of Health and Human Performance, Dublin City University（都柏林城市大学健康与人体运动学院）
In this paper we present an interactive tool that can be used to quantify fat infiltration in lumbar muscles, which is useful in studying fat infiltration and lower back pain (LBP) in adults. Currently, a qualitative assessment by visual grading via a 5-point scale is used to study fat infiltration in lumbar muscles from an axial view of lumbar-spine MR Images. However, a quantitative approach (on a continuous scale of 0-100\%) may provide a greater insight. In this paper, we propose a method to precisely quantify the fat deposition / infiltration in a user-defined region of the lumbar muscles…
Quantitative computed tomography (QCT) is a standard method to determine bone mineral density (BMD) in the spine. Traditionally single 8 - 10 mm thick slices have been analyzed only. Current spiral CT scanners provide true 3D acquisition schemes allowing for a more differential BMD analysis and an assessment of geometric parameters, which may improve fracture prediction. We developed a novel 3D segmentation approach that combines deformable balloons, multi seeded volume growing, and dedicated morphological operations to extract the vertebral bodies. An anatomy-oriented coordinate system attach…
The RSNA Lumbar Degenerative Imaging Spine Classification (LumbarDISC) Dataset
University of Utah School of Medicine（犹他大学医学院）；Thomas Jefferson University（托马斯·杰斐逊大学）；St. Michael’s Hospital（圣迈克尔医院）；Unity Health Toronto（多伦多联合医疗）；The Ohio State University（俄亥俄州立大学）；The Jackson Laboratory（杰克逊实验室）；Bunkerhilll Health（邦克希尔健康公司）；Universidade Federal de São Paulo（圣保罗联邦大学）；University of California San Francisco（加州大学旧金山分校）；Radiological Society of North America（北美放射学会）；Chiang Mai University Faculty of Medicine（清迈大学医学院）；Koc University School of Medicine（科奇大学医学院）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Diagnósticos da América S.A.（美洲诊断公司）；Gold Coast University Hospital（黄金海岸大学医院）；Clinical Center University of Sarajevo（萨拉热窝大学临床中心）；University Hospitals Cleveland Medical Center（克利夫兰大学医院医疗中心）；Tallaght University Hospital（塔拉特大学医院）；Kingston Health Sciences Centre（金斯顿健康科学中心）；Queen’s University（女王大学）；Loyola University Medical Center（洛约拉大学医学中心）；George Washington University Hospital（乔治华盛顿大学医院）；Children’s National Hospital（国家儿童医院）；Scripps Clinical Medical Group（斯克里普斯临床医疗集团）
SpineReport: Automated 3D Quantification and Reporting of Lumbar Spine Degeneration on MRI
NeuroPoly, Institute of Biomedical Engineering, Polytechnique Montreal（NeuroPoly，生物医学工程研究所，蒙特利尔大学）；Mila, Quebec AI Institute, Montreal（Mila，魁北克人工智能研究所，蒙特利尔）；Department of Radiology, Balgrist University Hospital, Zurich（放射科，巴尔格里斯大学医院，苏黎世）；Faculty of Medicine, University of Zurich（医学学院，苏黎世大学）；Department of Neurosurgery, Brain & Spine, University of South Florida（神经外科，脑与脊柱，佛罗里达大学）；Center for Neuromusculoskeletal Research, University of South Florida（神经肌肉研究中心，佛罗里达大学）；Department of Medical Engineering, University of South Florida（医学工程系，佛罗里达大学）；Stanford School of Medicine, Stanford（斯坦福医学院，斯坦福）；Functional Neuroimaging Unit, CRIUGM, Université de Montréal（功能神经影像单元，CRIUGM，蒙特利尔大学）；Centre de recherche du CHU Sainte-Justine, Université de Montréal（圣若泽中心研究，蒙特利尔大学）
Landmark Localization plays a very important role in processing medical images as well as in disease identification. However, In medical field, it's a challenging task because of the complexity of medical images and the high requirement of accuracy for disease identification and treatment.There are two dominant ways to regress landmark coordination, one using the full convolutional network to regress the heatmaps of landmarks , which is a complex way and heatmap post-process strategies are needed, and the other way is to regress the coordination using CNN + Full Connective Network directly, wh…
Anatomy-Aware Text-Visual Fusion with Dual-Perspective Prompts for Fine-Grained Lumbar Spine Segmentation
College of Computer and Data Science（计算机与数据科学学院）；Shandong Technology and Business University（山东科技商务大学）；Engineering Research Center of Big Data Intelligence (Fuzhou University)（大数据智能工程研究中心（福州大学））；The Chinese University of Hong Kong（香港中文大学）；Case Western Reserve University（凯斯西储大学）
Center for Data Science, Peking University（北京大学数据科学中心）；National Engineering Laboratory for Big Data Analysis and Applications, Peking University（北京大学大数据分析与应用国家工程实验室）；National Biomedical Imaging Center, Peking University（北京大学生物医学成像国家中心）
Accurate and efficient lumbar spine disease identification is crucial for clinical diagnosis. However, existing deep learning models with millions of parameters often fail to learn with only hundreds or dozens of medical images. These models also ignore the contextual relationship between adjacent objects, such as between vertebras and intervertebral discs. This work introduces a multi-scale context-guided network with coarse-to-fine localization and classification, named CCF-Net, for lumbar spine disease identification. Specifically, in learning, we divide the localization objective into two …
Klinikum rechts der Isar（伊萨尔右岸医院）；Technische Universität München（慕尼黑工业大学）
Multi-class segmentation of vertebrae is a non-trivial task mainly due to the high correlation in the appearance of adjacent vertebrae. Hence, such a task calls for the consideration of both global and local context. Based on this motivation, we propose a two-staged approach that, given a computed tomography dataset of the spine, segments the five lumbar vertebrae and simultaneously labels them. The first stage employs a multi-layered perceptron performing non-linear regression for locating the lumbar region using the global context. The second stage, comprised of a fully-convolutional deep ne…
In this paper we present a new 3D segmentation approach for the vertebrae of the lower thoracic and the lumbar spine in spiral computed tomography datasets. We implemented a multi-step procedure. Its main components are deformable models, volume growing, and morphological operations. The performance analysis that included an evaluation of accuracy using the European Spine Phantom, and of intra-operator precision using clinical CT datasets from 10 patients highlight the potential for clinical use. The intra-operator precision of the segmentation procedure was better than 1% for Bone Mineral Den…
Reconstruction of 3D lumbar spine models from incomplete segmentations using landmark detection
University of Koblenz（科布伦茨大学）；Institute for Computational Visualistics（计算视觉研究所）；Institute for Medical Technology and Information Processing（医学技术与信息处理研究所）
Registration plays an important role in medical image analysis. Deep learning-based methods have been studied for medical image registration, which leverage convolutional neural networks (CNNs) for efficiently regressing a dense deformation field from a pair of images. However, CNNs are limited in its ability to extract semantically meaningful intra- and inter-image spatial correspondences, which are of importance for accurate image registration. This study proposes a novel end-to-end deep learning-based framework for unsupervised affine and diffeomorphic deformable registration, referred as A…
University of Oxford（牛津大学）；Plexalis Ltd（Plexalis有限公司）
We propose a novel convolutional method for the detection and identification of vertebrae in whole spine MRIs. This involves using a learnt vector field to group detected vertebrae corners together into individual vertebral bodies and convolutional image-to-image translation followed by beam search to label vertebral levels in a self-consistent manner. The method can be applied without modification to lumbar, cervical and thoracic-only scans across a range of different MR sequences. The resulting system achieves 98.1% detection rate and 96.5% identification rate on a challenging clinical datas…
Hybrid Convolution Neural Network Integrated with Pseudo-Newton Boosting for Lumbar Spine Degeneration Detection
School of Computer Science and Engineering, Vellore Institute of Technology（韦洛尔理工大学计算机科学与工程学院）；Department of Information Science and Technology, College of Engineering Guindy, Anna University（安娜大学金迪工程学院信息科学与技术系）；Integrated Management Coastal Zones Research Institute, Universitat Politecnica de Valencia（瓦伦西亚理工大学海岸带综合管理研究所）
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）；University of Wisconsin, Madison（威斯康星大学麦迪逊分校）
The vertebral levels of the spine provide a useful coordinate system when making measurements of plaque, muscle, fat, and bone mineral density. Correctly classifying vertebral levels with high accuracy is challenging due to the similar appearance of each vertebra, the curvature of the spine, and the possibility of anomalies such as fractured vertebrae, implants, lumbarization of the sacrum, and sacralization of L5. The goal of this work is to develop a system that can accurately and robustly identify the L1 level in large heterogeneous datasets. The first approach we study is using a 3D U-Net …
Lumbar Spine Tumor Segmentation and Localization in T2 MRI Images Using AI
A. K. Choudhury School of Information Technology, University of Calcutta（加尔各答大学A.K.乔杜里信息技术学院）；Rammohan College, University of Calcutta（加尔各答大学拉莫汉学院）；Indian Statistical Institute（印度统计研究院）
Ambiguity is inevitable in medical images, which often results in different image interpretations (e.g. object boundaries or segmentation maps) from different human experts. Thus, a model that learns the ambiguity and outputs a probability distribution of the target, would be valuable for medical applications to assess the uncertainty of diagnosis. In this paper, we propose a powerful generative model to learn a representation of ambiguity and to generate probabilistic outputs. Our model, named Coordinate Quantization Variational Autoencoder (CQ-VAE) employs a discrete latent space with an int…
Huazhong University of Science and Technology（华中科技大学）；Wuhan National Laboratory for Optoelectronics（武汉光电国家研究中心）；City University of Hong Kong（香港城市大学）；The Hong Kong University of Science and Technology（香港科技大学）；Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属协和医院）
Vertebral landmark localization is a crucial step for variant spine-related clinical applications, which requires detecting the corner points of 17 vertebrae. However, the neighbor landmarks often disturb each other for the homogeneous appearance of vertebrae, which makes vertebral landmark localization extremely difficult. In this paper, we propose multi-stage cascaded convolutional neural networks (CNNs) to split the single task into two sequential steps, i.e., center point localization to roughly locate 17 center points of vertebrae, and corner point localization to find 4 corner points for…
Submillimeter-Accurate 3D Lumbar Spine Reconstruction from Biplanar X-Ray Images: Incorporating a Multi-Task Network and Landmark-Weighted Loss
School of Biomedical Engineering & Med-X Research Institute, Shanghai Jiao Tong University（上海交通大学生物医学工程学院与Med-X研究院）；Engineering Research Center of Digital Medicine and Clinical Translation, Ministry of Education（教育部数字医学临床转化工程研究中心）；Department of Spine Surgery, Tongji Hospital, School of Medicine, Tongji University（同济大学医学院附属同济医院脊柱外科）；TAOiMAGE Medical Technologies Corporation（TAOiMAGE医疗科技有限公司）；School of Mechanical Engineering, Shanghai Jiao Tong University（上海交通大学机械工程学院）
In this paper we introduce the Ladder Algorithm; a novel recurrent algorithm to detect repetitive structures in natural images with high accuracy using little training data. We then demonstrate the algorithm on the task of extracting vertebrae from whole spine magnetic resonance scans with only lumbar MR scans for training data. It is shown to achieve high perforamance with 99.8% precision and recall, exceeding current state of the art approaches for lumbar vertebrae detection in T1 and T2 weighted scans. It also generalises without retraining to whole spine images with minimal drop in accurac…
SpineMamba: Enhancing 3D Spinal Segmentation in Clinical Imaging through Residual Visual Mamba Layers and Shape Priors
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Northeast Petroleum University（东北石油大学）；Northeastern University（东北大学）；Southern Medical University（南方医科大学）
Adaptive Transformer Attention and Multi-Scale Fusion for Spine 3D Segmentation
University of Houston（休斯顿大学）；New York University（纽约大学）；University of Massachusetts Boston（马萨诸塞大学波士顿分校）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Tulane University（杜兰大学）
ARNAI: Artifact Removal Network based on Autoencoding and Inpainting for Robust Spinal Image Segmentation and Measurement
Pohang Stroke and Spine Hospital（浦项中风与脊柱医院）；Handong Global University（韩东全球大学）；Pohang University of Science and Technology（浦项科技大学）；Kyungpook National University Hospital（庆北国立大学医院）
Image segmentation in the medical domain is a challenging field owing to poor resolution and limited contrast. The predominantly used conventional segmentation techniques and the thresholding methods suffer from limitations because of heavy dependence on user interactions. Uncertainties prevalent in an image cannot be captured by these techniques. The performance further deteriorates when the images are corrupted by noise, outliers and other artifacts. The objective of this paper is to develop an effective robust fuzzy C- means clustering for segmenting vertebral body from magnetic resonance i…
Brigham and Women’s Hospital（布列根和妇女医院）；Massachusetts General Hospital（麻省总医院）
The high prevalence of spinal stenosis results in a large volume of MRI imaging, yet interpretation can be time-consuming with high inter-reader variability even among the most specialized radiologists. In this paper, we develop an efficient methodology to leverage the subject-matter-expertise stored in large-scale archival reporting and image data for a deep-learning approach to fully-automated lumbar spinal stenosis grading. Specifically, we introduce three major contributions: (1) a natural-language-processing scheme to extract level-by-level ground-truth labels from free-text radiology rep…
Southern Medical University（南方医科大学）；Western University（韦仕敦大学）；Digital Imaging Group of London（伦敦数字影像集团）
Automated quantitative measurement of the spine (i.e., multiple indices estimation of heights, widths, areas, and so on for the vertebral body and disc) is of the utmost importance in clinical spinal disease diagnoses, such as osteoporosis, intervertebral disc degeneration, and lumbar disc herniation, yet still an unprecedented challenge due to the variety of spine structure and the high dimensionality of indices to be estimated. In this paper, we propose a novel cascade amplifier regression network (CARN), which includes the CARN architecture and local shape-constrained manifold regularizatio…
SpineBench: Benchmarking Multimodal LLMs for Spinal Pathology Analysis
Beijing University of Posts and Telecommunications（北京邮电大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Peking Union Medical College Hospital（北京佑安医学大学医院）
The Hong Kong Polytechnic University（香港理工大学）；University of Oxford（牛津大学）
Automatic medical image segmentation based on Computed Tomography (CT) has been widely applied for computer-aided surgery as a prerequisite. With the development of deep learning technologies, deep convolutional neural networks (DCNNs) have shown robust performance in automated semantic segmentation of medical images. However, semantic segmentation algorithms based on DCNNs still meet the challenges of feature loss between encoder and decoder, multi-scale object, restricted field of view of filters, and lack of medical image data. This paper proposes a novel algorithm for automated vertebrae s…
University of Science and Technology of China（中国科学技术大学）；Guangzhou First People’s Hospital（广州市第一人民医院）；South China University of Technology（华南理工大学）
In the paper, we present an approach for learning a single model that universally segments 33 anatomical structures, including vertebrae, pelvic bones, and abdominal organs. Our model building has to address the following challenges. Firstly, while it is ideal to learn such a model from a large-scale, fully-annotated dataset, it is practically hard to curate such a dataset. Thus, we resort to learn from a union of multiple datasets, with each dataset containing the images that are partially labeled. Secondly, along the line of partial labelling, we contribute an open-source, large-scale verteb…
Machine learning technologies using deep neural networks (DNNs), especially convolutional neural networks (CNNs), have made automated, accurate, and fast medical image analysis a reality for many applications, and some DNN-based medical image analysis systems have even been FDA-cleared. Despite the progress, challenges remain to build DNNs as reliable as human expert doctors. It is known that DNN classifiers may not be robust to noises: by adding a small amount of noise to an input image, a DNN classifier may make a wrong classification of the noisy image (i.e., in-distribution adversarial sam…
University of Maryland（马里兰大学）；PAII Inc.（PAII公司）；Alibaba Group DAMO Academy（阿里巴巴集团达摩院）；Xpeng Motors（小鹏汽车）；Ping An Technology（平安科技）；Chang Gung Memorial Hospital（长庚纪念医院）
Osteoporosis is a common chronic metabolic bone disease often under-diagnosed and under-treated due to the limited access to bone mineral density (BMD) examinations, e.g. via Dual-energy X-ray Absorptiometry (DXA). This paper proposes a method to predict BMD from Chest X-ray (CXR), one of the most commonly accessible and low-cost medical imaging examinations. Our method first automatically detects Regions of Interest (ROIs) of local CXR bone structures. Then a multi-ROI deep model with transformer encoder is developed to exploit both local and global information in the chest X-ray image for ac…
From Baseline to Follow-Up: Counterfactual Spine DXA Image Synthesis in UK Biobank Using a Causal Hierarchical Variational Autoencoder
School of Electronics and Computer Science（电子与计算机科学学院）；University of Southampton（萨塞克斯大学）；MRC Lifecourse Epidemiology Centre（英国医学研究理事会生命周期流行病学中心）；University of Southampton, Southampton General Hospital（萨塞克斯大学索马塞特医院）；Computer Science University of Southampton（计算机科学萨塞克斯大学）
Domain adaptation (DA) has drawn high interests for its capacity to adapt a model trained on labeled source data to perform well on unlabeled or weakly labeled target data from a different domain. Most common DA techniques require the concurrent access to the input images of both the source and target domains. However, in practice, it is common that the source images are not available in the adaptation phase. This is a very frequent DA scenario in medical imaging, for instance, when the source and target images come from different clinical sites. We propose a novel formulation for adapting seg…
National Institutes of Health（美国国立卫生研究院）；University of California-Irvine（加利福尼亚大学尔湾分校）
Accurate spine segmentation allows for improved identification and quantitative characterization of abnormalities of the vertebra, such as vertebral fractures. However, in existing automated vertebra segmentation methods on computed tomography (CT) images, leakage into nearby bones such as ribs occurs due to the close proximity of these visibly intense structures in a 3D CT volume. To reduce this error, we propose the use of joint vertebra-rib atlases to improve the segmentation of vertebrae via multi-atlas joint label fusion. Segmentation was performed and evaluated on CTs containing 106 thor…
Domain adaptation (DA) has drawn high interest for its capacity to adapt a model trained on labeled source data to perform well on unlabeled or weakly labeled target data from a different domain. Most common DA techniques require concurrent access to the input images of both the source and target domains. However, in practice, privacy concerns often impede the availability of source images in the adaptation phase. This is a very frequent DA scenario in medical imaging, where, for instance, the source and target images could come from different clinical sites. We introduce a source-free domain …
