Augmented reality has the potential to improve operating room workflow by allowing physicians to "see" inside a patient through the projection of imaging directly onto the surgical field. For this to be useful the acquired imaging must be quickly and accurately registered with patient and the registration must be maintained. Here we describe a method for projecting a CT scan with Microsoft Hololens and then aligning that projection to a set of fiduciary markers. Radio-opaque stickers with unique QR-codes are placed on an object prior to acquiring a CT scan. The location of the markers in the CT scan are extracted and the CT scan is converted into a 3D surface object. The 3D object is then projected using the Hololens onto a table on which the same markers are placed. We designed an algorithm that aligns the markers on the 3D object with the markers on the table. To extract the markers and convert the CT into a 3D object took less than 5 seconds. To align three markers, it took $0.9 \pm 0.2$ seconds to achieve an accuracy of $5 \pm 2$ mm. These findings show that it is feasible to use a combined radio-opaque optical marker, placed on a patient prior to a CT scan, to subsequently align the acquired CT scan with the patient.
Towards Markerless Intraoperative Tracking of Deformable Spine Tissue
The Hamlyn Centre for Robotic Surgery, Imperial College London, London, UK, Department of Computing, Imperial College London, London, UK Department of Mechanical Engineering, Imperial College London, London, UK Department of Surgery \& Cancer, Imperial College London, London, UK Department of Neurosurgery, IRCCS Humanitas Research Hospital, Rozzano, Lombardy, Italy Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Milan, Italy. Stanford University School of Medicine, Stanford University, California, USA
Monocular Marker-free Patient-to-Image Intraoperative Registration for Cochlear Implant Surgery
Dept. of Computer Science, Vanderbilt University（计算机科学系，范德比尔特大学）；Dept. of Electrical and Computer Engineering, Vanderbilt University（电气与计算机工程系，范德比尔特大学）
PIVOTS: Aligning unseen Structures using Preoperative to Intraoperative Volume-To-Surface Registration for Liver Navigation
Translational Surgical Oncology, National Center for Tumor Diseases（转化外科肿瘤学，肿瘤疾病国家中心）；German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Faculty of Medicine and University Hospital Carl Gustav Carus（医学系和卡尔·戈斯塔·卡尔斯大学医院）；Department of Visceral, Thoracic and Vascular Surgery, University Hospital and Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology（visceral、胸腔和血管外科部门，大学医院和医学系卡尔·戈斯塔·卡尔斯，德累斯顿技术大学）；Centre for Tactile Internet with Human-in-the-Loop, TU Dresden（人机交互触觉互联网中心，德累斯顿技术大学）；School of Control Science and Engineering, Shandong University（控制科学与工程学院，山东大学）；Surgical Department, The Netherlands Cancer Institute（手术部，荷兰癌症研究所）；Translational Imaging in Oncology, National Center for Tumor Diseases（肿瘤转化影像学，肿瘤疾病国家中心）；Weldon School of Biomedical Engineering, Purdue University（韦尔登生物医学工程学院，普渡大学）
Bridging the Markerless Tracking Validation Gap: Intraoperative X-ray Workflow Integration
The Hamlyn Centre for Robotic Surgery, Imperial College London（伦敦帝国理工学院哈姆林外科机器人中心）；Imperial College Healthcare NHS Trust（帝国理工学院医疗保健 NHS 信托基金）；Stanford University School of Medicine（斯坦福大学医学院）
LayersReg: A Layer-by-Layer Progressive Regressor for Reliable Intraoperative 3D/2D Registration
School of Biological Science and Medical Engineering（生物科学与医学工程学院）；Southeast University（东南大学）；School of Engineering, College of Engineering and Physical Sciences（工程学院，工程与物理科学学院）；University of Birmingham（伯明翰大学）；State Key Laboratory of Digital Medical Engineering（数字医学工程国家重点实验室）
Landmark-Free Preoperative-to-Intraoperative Registration in Laparoscopic Liver Resection
Center of Smart Health, School of Nursing, The Hong Kong Polytechnic University（智能健康中心、护理学院、香港理工大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Institute of Medical Intelligence and XR（医学智能与XR研究所）；The Chinese University of Hong Kong（香港中文大学）；Division of Hepatobiliopancreatic Surgery, Department of General Surgery（肝胆胰外科 division, 外科部）
Balgrist University Hospital（巴尔格里斯特大学医院）；ETH Zurich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）
In surgical navigation, finding correspondence between preoperative plan and intraoperative anatomy, the so-called registration task, is imperative. One promising approach is to intraoperatively digitize anatomy and register it with the preoperative plan. State-of-the-art commercial navigation systems implement such approaches for pedicle screw placement in spinal fusion surgery. Although these systems improve surgical accuracy, they are not gold standard in clinical practice. Besides economical reasons, this may be due to their difficult integration into clinical workflows and unintuitive nav…
Unsupervised MR-US Multimodal Image Registration with Multilevel Correlation Pyramidal Optimization
School of Artificial Intelligence and Robotics, Hunan University, Changsha, Hunan, China（人工智能与机器人学院，湖南大学，长沙，湖南，中国）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, Hunan, China（机器人视觉感知与控制技术国家工程研究中心，湖南大学，长沙，湖南，中国）；National University of Singapore（新加坡国立大学）；Cornell University（康奈尔大学）
We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive …
Periacetabular osteotomy is a challenging surgical procedure for treating developmental hip dysplasia, providing greater coverage of the femoral head via relocation of a patient's acetabulum. Since fluoroscopic imaging is frequently used in the surgical workflow, computer-assisted X-Ray navigation of osteotomes and the relocated acetabular fragment should be feasible. We use intensity-based 2D/3D registration to estimate the pelvis pose with respect to fluoroscopic images, recover relative poses of multiple views, and triangulate landmarks which may be used for navigation. Existing similarity …
Resolving the Ambiguity of Complete-to-Partial Point Cloud Registration for Image-Guided Liver Surgery with Patches-to-Partial Matching
Center for Imaging Science and Department of Biomedical Engineering, Rochester Institute of Technology（成像科学中心和生物医学工程系，罗切斯特理工学院）；Department of Biomedical Engineering, Vanderbilt University（生物医学工程系，范德比尔特大学）；School of Science and Engineering, University of Missouri – Kansas City（科学与工程学院，密苏里大学-堪萨斯城分校）
Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization
College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学机器人视觉感知与控制技术国家工程研究中心）；Cornell University（康奈尔大学）
Optimization-Based Calibration for Intravascular Ultrasound Volume Reconstruction
Inria；University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；INSERM（法国国家医学研究院）；ICube（ICube研究中心）；UMR7357（UMR7357研究单位）；University of Basel（巴塞尔大学）；Department of Biomedical Engineering（生物医学工程系）；Institute of Image-Guided Surgery（影像引导手术研究所）；IHU Strasbourg（斯特拉斯堡IHU）
Robust and Accurate Multi-view 2D/3D Image Registration with Differentiable X-ray Rendering and Dual Cross-view Constraints
Department of Information Science and Engineering, Shandong University（信息科学与工程学院，山东大学）；School of Control Science and Engineering, Shangdong University（控制科学与工程学院，山东大学）；Shenzhen Key Laboratory of Robotics Perception and Intelligence（机器人感知与智能实验室）；Dept. of Electronic and Electrical Engineering, Southern University of Science and Technology（电子与电气工程系，南方科技大学）；Dept. of Electronic Engineering, The Chinese University of Hong Kong（电子工程系，香港中文大学）
College of Science, Beijing Forestry University（北京林业大学理学院）；Academy of Opto-Electronics（光电研究院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）
Point-cloud data acquired using a terrestrial laser scanner (TLS) play an important role in digital forestry research. Multiple scans are generally used to overcome occlusion effects and obtain complete tree structural information. However, it is time-consuming and difficult to place artificial reflectors in a forest with complex terrain for marker-based registration, a process that reduces registration automation and efficiency. In this study, we propose an automatic coarse-to-fine method for the registration of point-cloud data from multiple scans of a single tree. In coarse registration, po…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）；Auris Health, Inc.（奥里斯健康公司）；Johns Hopkins Medicine（约翰斯·霍普金斯医学院）；Nara Institute of Science and Technology（奈良科学技术研究所）；University of Texas（德克萨斯大学）；Texas Orthopedics（德克萨斯骨科医院）
Fluoroscopy is the standard imaging modality used to guide hip surgery and is therefore a natural sensor for computer-assisted navigation. In order to efficiently solve the complex registration problems presented during navigation, human-assisted annotations of the intraoperative image are typically required. This manual initialization interferes with the surgical workflow and diminishes any advantages gained from navigation. We propose a method for fully automatic registration using annotations produced by a neural network. Neural networks are trained to simultaneously segment anatomy and ide…
End2Reg: Learning Task-Specific Segmentation for Markerless Registration in Spine Surgery
Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland（巴塞尔大学生物医学工程系，瑞士Allschwil）；Department of Orthopedics, University Children’s Hospital, Basel, Switzerland（巴塞尔大学儿童医院骨科部，瑞士Basel）
Rapid patient-specific neural networks for intraoperative X-ray to volume registration
Harvard-MIT Health Sciences and Technology, Massachusetts Institute of Technology（哈佛-麻省理工健康科学与技术, 麻省理工学院）；Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology（计算机科学与人工智能实验室, 麻省理工学院）；Department of Radiology, Harvard Medical School（哈佛医学院放射科）；Saint Luke’s Marion Bloch Neuroscience Institute（圣路易斯马里恩布洛克神经科学研究所）；Department of Critical Care Medicine, Shriners Children’s Hospital（谢尔曼儿童医院重症医学科）；Department of Interventional Neuroradiology, Boston Children’s Hospital（波士顿儿童医院介入神经放射科）；Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（阿提努拉A·马丁诺斯生物医学成像中心, 麻省总医院）
Many applications including object reconstruction, robot guidance, and scene mapping require the registration of multiple views from a scene to generate a complete geometric and appearance model of it. In real situations, transformations between views are unknown an it is necessary to apply expert inference to estimate them. In the last few years, the emergence of low-cost depth-sensing cameras has strengthened the research on this topic, motivating a plethora of new applications. Although they have enough resolution and accuracy for many applications, some situations may not be solved with ge…
Towards Patient-Specific Deformable Registration in Laparoscopic Surgery
Biomedical Robotics Lab, Istituto Italiano di Tecnologia, Genoa, Italy（生物医学机器人实验室，意大利理工学院，热那亚）；Department of Computer Science, Bioengineering, Robotics and Systems Engineering (DIBRIS), University of Genoa, Italy（计算机科学、生物工程、机器人与系统工程系（DIBRIS），热那亚大学，意大利）；Harvard Medical School, Brigham and Women’s Hospital, Boston, USA（哈佛医学院，布里洛妇女医院，美国波士顿）
Harvard-MIT Health Sciences and Technology（哈佛-麻省理工卫生科学与技术部）；Massachusetts Institute of Technology（麻省理工学院）；Computer Science and Artificial Intelligence Laboratory（计算机科学与人工智能实验室）
The use of digitally reconstructed radiographs (DRRs) to solve inverse problems such as slice-to-volume registration and 3D reconstruction is well-studied in preoperative settings. In intraoperative imaging, the utility of DRRs is limited by the challenges in generating them in real-time and supporting optimization procedures that rely on repeated DRR synthesis. While immense progress has been made in accelerating the generation of DRRs through algorithmic refinements and GPU implementations, DRR-based optimization remains slow because most DRR generators do not offer a straightforward way to …
Automatic Multi-View X-Ray/CT Registration Using Bone Substructure Contours
University Children’s Hospital Zürich（苏黎世儿童医院大学医院）；University Hospital Balgrist, University of Zurich（苏黎世大学医院巴尔格里斯分院）；Department of Orthopedic Surgery（骨科部门）
What neurosurgeons need to see: synthetic intra-operative MRI from ultrasound for brain-shift compensation in brain tumour surgery
Department of Neurosurgery, Neurovascular Unit, Río Hortega University Hospital, Valladolid, Spain（西班牙巴利亚多利德里奥·奥尔特加大学医院神经外科神经血管科）；Specialized Group in Biomedical Imaging and Computational Analysis (GEIBAC), Instituto de Investigación Biosanitaria de Valladolid (IBioVALL), Valladolid, Spain（西班牙巴利亚多利德生物医学研究与计算分析专业组(GEIBAC)，巴利亚多利德生物健康研究所(IBioVALL)）
Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy
AI in Medicine and Surgery Group, School of Computer Science, University of Leeds（利兹大学计算机学院医学与外科人工智能小组）；Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院放射诊断学系）；Department of HPB and Transplant Surgery, St. James’s University Hospital（圣詹姆斯大学医院肝脏胰胆和移植外科）
The Brain Resection Multimodal Image Registration (ReMIND2Reg) 2025 Challenge
Inria Saclay Île-de-France（法国里昂萨克利研究所）；CEA（法国原子能委员会）；Université Paris-Saclay（巴黎-萨克利大学）；Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（巴黎脑研究所）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化技术研究所）；Inserm（法国国家医学研究院）；AP-HP（法国国家医院集团）；Hôpital de la Pitié Salpêtrière（皮蒂埃-萨尔普里埃尔医院）；Harvard Medical School（哈佛医学院）；Brigham and Women's Hospital（布里根医院）；Department of Radiology and Radiological Science（放射学与放射科学系）；Johns Hopkins Medical School（约翰霍普金斯医学院）；Institute of Medical Informatics（医学信息学研究所）；University of Lübeck（吕贝克大学）；Image Analysis and Communications Laboratory（图像分析与通信实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Johns Hopkins University（约翰霍普金斯大学）；CSAIL（媒体实验室）
Markerless Tracking-Based Registration for Medical Image Motion Correction
Department Artificial Intelligence in Biomedical Engineering, FAU Erlangen-Nürnberg（生物医学工程人工智能系，埃尔朗根-纽伦堡大学）；Department of Communication Sciences and Disorders, East Carolina University（通讯科学与障碍系，东卡罗来纳大学）；Department of Otolaryngology–Head and Neck Surgery, Louisiana State University Health Sciences Center（耳鼻喉科与头颈外科系，路易斯安那州立大学健康科学中心）；Department of Radiology, FAU Erlangen-Nürnberg（放射学系，埃尔朗根-纽伦堡大学）；Department of Communication Sciences and Disorders, Louisiana State University（通讯科学与障碍系，路易斯安那州立大学）
WarpEM: Dynamic Time Warping for Accurate Catheter Registration in EM-guided Procedures
Technical University of Munich（慕尼黑工业大学）；German Heart Center Munich（慕尼黑德国心脏中心）；German Centre for Cardiovascular Research（德国心血管研究中心）；Johns Hopkins University（约翰斯·霍普金斯大学）
An objective comparison of methods for augmented reality in laparoscopic liver resection by preoperative-to-intraoperative image fusion
University of Leeds（利兹大学）；Centre Hospitalier Universitaire de Clermont-Ferrand（克莱蒙费朗大学医院中心）；National University of Singapore (NUS)（新加坡国立大学）；National Center for Tumor Diseases (NCT/UCC Dresden)（德国国家肿瘤疾病中心（德累斯顿））；Fudan University（复旦大学）；University of Pennsylvania（宾夕法尼亚大学）
Technion – Israel Institute of Technology（以色列理工学院）
Quantification of uncertainty in deep-neural-networks (DNN) based image registration algorithms plays a critical role in the deployment of image registration algorithms for clinical applications such as surgical planning, intraoperative guidance, and longitudinal monitoring of disease progression or treatment efficacy as well as in research-oriented processing pipelines. Currently available approaches for uncertainty estimation in DNN-based image registration algorithms may result in sub-optimal clinical decision making due to potentially inaccurate estimation of the uncertainty of the registr…
We present a meta-learning framework for interactive medical image registration. Our proposed framework comprises three components: a learning-based medical image registration algorithm, a form of user interaction that refines registration at inference, and a meta-learning protocol that learns a rapidly adaptable network initialization. This paper describes a specific algorithm that implements the registration, interaction and meta-learning protocol for our exemplar clinical application: registration of magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled transrectal ult…
School of Geospatial Engineering and Science, Sun Yat-Sen University（中山大学测绘科学与技术学院）；Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University（香港理工大学土地测量及地理资讯学系）；Changjiang River Scientific Research Institute (CRSRI)（长江水利委员会长江科学院）；Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院数字地球重点实验室）；Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)（南方海洋科学与工程广东省实验室（珠海））
Registration of unmanned aerial vehicle laser scanning (ULS) and ground light detection and ranging (LiDAR) point clouds in forests is critical to create a detailed representation of a forest structure and an accurate inversion of forest parameters. However, forest occlusion poses challenges for marker-based registration methods, and some marker-free automated registration methods have low efficiency due to the process of object (e.g., tree, crown) segmentation. Therefore, we use a divide-and-conquer strategy and propose an automated and efficient method to register ULS and ground LiDAR point …
Respiratory Differencing: Enhancing Pulmonary Thermal Ablation Evaluation Through Pre- and Intra-Operative Image Fusion
Beijing Institute of Respiratory Medicine（北京呼吸病研究所）；Beijing Chao Yang Hospital（北京朝阳医院）；Capital Medical University（首都医科大学）；First Imaging Medical Equipment（第一影像医疗设备公司）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）
University of Utah（犹他大学）；University of Maryland, School of Medicine（马里兰大学医学院）
Radiation therapy presents a need for dynamic tracking of a target tumor volume. Fiducial markers such as implanted gold seeds have been used to gate radiation delivery but the markers are invasive and gating significantly increases treatment time. Pretreatment acquisition of a respiratory correlated 4DCT allows for determination of accurate motion tracking which is useful in treatment planning. We design a patient-specific motion subspace and a deep convolutional neural network to recover anatomical positions from a single fluoroscopic projection in real-time. We use this deep network to appr…
Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR
UCL Hawkes Institute, University College London, London WC1E 6BT, UK（伦敦大学学院UCL哈维斯研究所）；Division of Surgery and Interventional Science, University College London, London WC1E 6BT, UK（伦敦大学学院UCL外科与介入科学系）；Unit for Lifelong Health and Ageing at UCL, University College London, London WC1E 7HB, UK（伦敦大学学院UCL终身健康与老龄化单位）；Medtronic plc., London, UK（伦敦梅脱利克公司）
