End2Reg: Learning Task-Specific Segmentation for Markerless Registration in Spine Surgery
Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland（巴塞尔大学生物医学工程系，瑞士Allschwil）；Department of Orthopedics, University Children’s Hospital, Basel, Switzerland（巴塞尔大学儿童医院骨科部，瑞士Basel）
Lorenzo Pettinari, Sidaty El Hadramy, Michael Wehrli, Philippe C. Cattin, Daniel Studer, Carol C. Hasler, Maria Licci
Intraoperative navigation in spine surgery demands millimeter-level accuracy. Currently, this is achieved through radiation-intensive intraoperative imaging and bone-anchored markers that are invasive and disrupt surgical workflow. Markerless RGB-D registration methods offer a promising alternative. However, existing approaches rely on weak segmentation labels to isolate relevant anatomical structures, potentially propagating errors through the registration process. We present End2Reg, an end-to-end deep learning framework that jointly optimizes segmentation and registration, eliminating the need for segmentation labels and manual steps. The network learns task-specific segmentation masks optimized for registration, guided solely by the registration objective without explicit segmentation supervision. End2Reg achieves state-of-the-art performance on ex- and in-vivo benchmarks, reducing median Target Registration Error by 32% and mean Root Mean Square Error by 61%, while maintaining robust performance under partial occlusions. Ablation results confirm that end-to-end optimization significantly improves registration accuracy. Overall, End2Reg advances towards fully automatic, markerless intraoperative navigation. Code and interactive visualizations are available at: https://lorenzopettinari.github.io/end-2-reg/.
Towards Markerless Intraoperative Tracking of Deformable Spine Tissue
The Hamlyn Centre for Robotic Surgery, Imperial College London, London, UK, Department of Computing, Imperial College London, London, UK Department of Mechanical Engineering, Imperial College London, London, UK Department of Surgery \& Cancer, Imperial College London, London, UK Department of Neurosurgery, IRCCS Humanitas Research Hospital, Rozzano, Lombardy, Italy Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Milan, Italy. Stanford University School of Medicine, Stanford University, California, USA
Suture needle localization is necessary for autonomous suturing. Previous approaches in autonomous suturing often relied on fiducial markers rather than markerless detection schemes for localizing a suture needle due to the inconsistency of markerless detections. However, fiducial markers are not practical for real-world applications and can often be occluded from environmental factors in surgery (e.g., blood). Therefore in this work, we present a robust tracking approach for estimating the 6D pose of a suture needle when using inconsistent detections. We define observation models based on sut…
Conventional computer-assisted orthopaedic navigation systems rely on the tracking of dedicated optical markers for patient poses, which makes the surgical workflow more invasive, tedious, and expensive. Visual tracking has recently been proposed to measure the target anatomy in a markerless and effortless way, but the existing methods fail under real-world occlusion caused by intraoperative interventions. Furthermore, such methods are hardware-specific and not accurate enough for surgical applications. In this paper, we propose a RGB-D sensing-based markerless tracking method that is robust a…
Asan Medical Center（峨山医疗中心）；Johns Hopkins University（约翰斯·霍普金斯大学）；Chungnam National University School of Medicine（忠南大学医学院）
Some endoscopic surgeries require a surgeon to hold the endoscope with one hand and the surgical instruments with the other hand to perform the actual surgery with correct vision. Recent technical advances in deep learning as well as in robotics can introduce robotics to these endoscopic surgeries. This can have numerous advantages by freeing one hand of the surgeon, which will allow the surgeon to use both hands and to use more intricate and sophisticated techniques. Recently, deep learning with convolutional neural network achieves state-of-the-art results in computer vision. Therefore, the …
Fiducial markers are commonly used in navigation assisted minimally invasive spine surgery (MISS) and they help transfer image coordinates into real world coordinates. In practice, these markers might be located outside the field-of-view (FOV), due to the limited detector sizes of C-arm cone-beam computed tomography (CBCT) systems used in intraoperative surgeries. As a consequence, reconstructed markers in CBCT volumes suffer from artifacts and have distorted shapes, which sets an obstacle for navigation. In this work, we propose two fiducial marker detection methods: direct detection from dis…
Polytechnique Montreal（蒙特利尔综合理工大学）；Sainte-Justine Hospital Research Center（Sainte-Justine医院研究中心）；University of Ljubljana（卢布尔雅那大学）
Anterior Vertebral Body Growth Modulation (AVBGM) is a minimally invasive surgical technique that gradually corrects spine deformities while preserving lumbar motion. However the selection of potential surgical patients is currently based on clinical judgment and would be facilitated by the identification of patients responding to AVBGM prior to surgery. We introduce a statistical framework for predicting the surgical outcomes following AVBGM in adolescents with idiopathic scoliosis. A discriminant manifold is first constructed to maximize the separation between responsive and non-responsive g…
Korea Institute of Science and Technology（韩国科学技术研究院）；Korea University of Science and Technology（韩国科学技术大学）；Korea Advanced Institute of Science and Technology（韩国科学技术院）
There has been a significant increase from 2010 to 2016 in the number of people suffering from spine problems. The automatic image segmentation of the spine obtained from a computed tomography (CT) image is important for diagnosing spine conditions and for performing surgery with computer-assisted surgery systems. The spine has a complex anatomy that consists of 33 vertebrae, 23 intervertebral disks, the spinal cord, and connecting ribs. As a result, the spinal surgeon is faced with the challenge of needing a robust algorithm to segment and create a model of the spine. In this study, we develo…
An Optimization-based Baseline for Rigid 2D/3D Registration Applied to Spine Surgical Navigation Using CMA-ES
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Southeast University（东南大学江苏省医学信息处理协同创新国际联合实验室）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education（教育部新一代人工智能技术及其交叉应用重点实验室（东南大学））
Charles Sturt University（查尔斯特大学）；Study Group Australia（澳大利亚学习集团）；University of Technology, Baghdad（巴格达技术大学）；University of Kurdistan Hewler（库尔德斯坦赫勒大学）
Background and aim: Image registration and alignment are the main limitations of augmented reality-based knee replacement surgery. This research aims to decrease the registration error, eliminate outcomes that are trapped in local minima to improve the alignment problems, handle the occlusion, and maximize the overlapping parts. Methodology: markerless image registration method was used for Augmented reality-based knee replacement surgery to guide and visualize the surgical operation. While weight least square algorithm was used to enhance stereo camera-based tracking by filling border occlusi…
Reconstruction of 3D lumbar spine models from incomplete segmentations using landmark detection
University of Koblenz（科布伦茨大学）；Institute for Computational Visualistics（计算视觉研究所）；Institute for Medical Technology and Information Processing（医学技术与信息处理研究所）
CTSpine1K: A Large-Scale Dataset for Spinal Vertebrae Segmentation in Computed Tomography
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所智能信息处理重点实验室）；Suzhou Institute of Intelligent Computing Technology, Chinese Academy of Sciences（中国科学院苏州智能计算技术研究院）；Beijing Jishuitan Hospital（北京积水潭医院）；School of Medicine, South China University of Technology（华南理工大学医学院）
SurgPointTransformer: Vertebrae Shape Completion with RGB-D Data
Research in Orthopedic Computer Science, University Hospital Balgrist, University of Zürich（苏黎世大学、巴尔格里斯特大学医院骨科计算机科学研究所）；Balgrist University Hospital, University of Zurich（苏黎世大学巴尔格里斯特大学医院）
Virtual-reality based patient-specific simulation of spine surgical procedures: A fast, highly automated and high-fidelity system for surgical education and planning
Holland Bone and Joint Program, Sunnybrook Research Institute（霍尔德骨科与关节程序，圣布鲁诺研究学院）；Division of Spine Surgery, Sunnybrook Health Sciences Centre（脊柱外科部，圣布鲁诺健康科学中心）；Division of Orthopaedic Surgery, Department of Surgery, University of Toronto（骨科部，外科部，多伦多大学）
NeuralBoneReg: An Instance-Specific Label-Free Point Cloud-Based Method for Multi-Modal Bone Surface Registration
Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich（骨科计算机科学研究所，巴尔格里斯大学医院，苏黎世大学）；AI Center, ETH Zurich（人工智能中心，苏黎世联邦理工学院）
Technical University of Munich（慕尼黑工业大学）；Klinikum Rechts der Isar（伊萨尔河右岸医院）；Helmholtz Zentrum München（亥姆霍兹慕尼黑中心）；Graz University of Technology（格拉茨工业大学）；Medical University of Graz（格拉茨医科大学）；The University of Auckland（奥克兰大学）；iFLYTEK Research（科大讯飞研究院）
Vertebral labelling and segmentation are two fundamental tasks in an automated spine processing pipeline. Reliable and accurate processing of spine images is expected to benefit clinical decision-support systems for diagnosis, surgery planning, and population-based analysis on spine and bone health. However, designing automated algorithms for spine processing is challenging predominantly due to considerable variations in anatomy and acquisition protocols and due to a severe shortage of publicly available data. Addressing these limitations, the Large Scale Vertebrae Segmentation Challenge (VerS…
Spinal Pain Research Institute of Tongji University（同济大学脊柱疼痛研究所）；Brigham and Women's Hospital, Harvard Medical School（哈佛医学院布莱根妇女医院）；School of Data and Computer Science, Sun Yat-sen University（中山大学数据科学与计算机学院）；Shanghai Jiao Tong University School of Medicine（上海交通大学医学院）；Graduate School of Frontier Sciences, The University of Tokyo（东京大学前沿科学研究生院）；Burke Neurological Institute（伯克神经学研究所）
Background and objective: Combined evaluation of lumbosacral structures (e.g. nerves, bone) on multimodal radiographic images is routinely conducted prior to spinal surgery and interventional procedures. Generally, magnetic resonance imaging is conducted to differentiate nerves, while computed tomography (CT) is used to observe bony structures. The aim of this study is to investigate the feasibility of automatically segmenting lumbosacral structures (e.g. nerves & bone) on non-contrast CT with deep learning. Methods: a total of 50 cases with spinal CT were manually labeled for lumbosacral nerv…
IXGS-Intraoperative 3D Reconstruction from Sparse, Arbitrarily Posed Real X-rays
Research in Orthopedic Computer Science, Balgrist University Hospital, Zurich, 8008, Switzerland（骨科计算机科学研究所，巴尔格里斯大学医院，苏黎世，8008，瑞士）；The Australian National University, Canberra, ACT 2601, Australia（澳大利亚国立大学，堪培拉，ACT 2601，澳大利亚）；Department of Orthopedics, Balgrist University Hospital, University of Zurich, Zurich, 8008, Switzerland（骨科系，巴尔格里斯大学医院，苏黎世大学，苏黎世，8008，瑞士）
