Image registration is a key operation in medical image processing, allowing a plethora of applications. Mutual information (MI) is consolidated as a robust similarity metric often used for medical image registration. Although MI provides a robust medical image registration, it usually fails when the needed image transform is too big due to MI local maxima traps. In this paper, we propose and evaluate a generalized parametric MI as an affine registration cost function. We assessed the generalized MI (GMI) functions for separable affine transforms and exhaustively evaluated the GMI mathematical image seeking the maximum registration range through a gradient descent simulation. We also employed Monte Carlo simulation essays for testing translation registering of randomized T1 versus T2 images. GMI functions showed to have smooth isosurfaces driving the algorithm to the global maxima. Results show significantly prolonged registration ranges, avoiding the traps of local maxima. We evaluated a range of [-150mm,150mm] for translations, [-180°,180°] for rotations, [0.5,2] for scales, and [-1,1] for skew with a success rate of 99.99%, 97.58%, 99.99%, and 99.99% respectively for the transforms in the simulated gradient descent. We also obtained 99.75% success in Monte Carlo simulation from 2,000 randomized translations trials with 1,113 subjects T1 and T2 MRI images. The findings point towards the reliability of GMI for long-range registration with enhanced speed performance
We present a robust method to correct for motion and deformations for in-utero volumetric MRI time series. Spatio-temporal analysis of dynamic MRI requires robust alignment across time in the presence of substantial and unpredictable motion. We make a Markov assumption on the nature of deformations to take advantage of the temporal structure in the image data. Forward message passing in the corresponding hidden Markov model (HMM) yields an estimation algorithm that only has to account for relatively small motion between consecutive frames. We demonstrate the utility of the temporal model by sh…
QUIZ: An Arbitrary Volumetric Point Matching Method for Medical Image Registration
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Guangdong Medical University（广东医科大学）
Shanghai Jiao Tong University（上海交通大学）；University of Sydney（悉尼大学）
Dynamic medical imaging is usually limited in application due to the large radiation doses and longer image scanning and reconstruction times. Existing methods attempt to reduce the dynamic sequence by interpolating the volumes between the acquired image volumes. However, these methods are limited to either 2D images and/or are unable to support large variations in the motion between the image volume sequences. In this paper, we present a spatiotemporal volumetric interpolation network (SVIN) designed for 4D dynamic medical images. SVIN introduces dual networks: first is the spatiotemporal mot…
Universidad Politécnica de Madrid（马德里理工大学）；Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN)（生物医学研究网络中心（生物工程、生物材料与纳米医学））；King’s College London（伦敦国王学院）
Magnetic resonance imaging of whole fetal body and placenta is limited by different sources of motion affecting the womb. Usual scanning techniques employ single-shot multi-slice sequences where anatomical information in different slices may be subject to different deformations, contrast variations or artifacts. Volumetric reconstruction formulations have been proposed to correct for these factors, but they must accommodate a non-homogeneous and non-isotropic sampling, so regularization becomes necessary. Thus, in this paper we propose a deep generative prior for robust volumetric reconstructi…
Massachusetts Institute of Technology（麻省理工学院）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；Washington University in St. Louis（圣路易斯华盛顿大学）；Shanghai Jiao Tong University（上海交通大学）
We present a robust method to correct for motion in volumetric in-utero MRI time series. Time-course analysis for in-utero volumetric MRI time series often suffers from substantial and unpredictable fetal motion. Registration provides voxel correspondences between images and is commonly employed for motion correction. Current registration methods often fail when aligning images that are substantially different from a template (reference image). To achieve accurate and robust alignment, we make a Markov assumption on the nature of motion and take advantage of the temporal smoothness in the imag…
Columbia University（哥伦比亚大学）；Columbia University Medical Center（哥伦比亚大学医学中心）；University of Iowa（爱荷华大学）
Unsupervised learning-based medical image registration approaches have witnessed rapid development in recent years. We propose to revisit a commonly ignored while simple and well-established principle: recursive refinement of deformation vector fields across scales. We introduce a recursive refinement network (RRN) for unsupervised medical image registration, to extract multi-scale features, construct normalized local cost correlation volume and recursively refine volumetric deformation vector fields. RRN achieves state of the art performance for 3D registration of expiratory-inspiratory pairs…
CentraleSupelec（中央理工-高等电力学院）；INRIA（法国国家信息与自动化研究所）；Universite Paris-Saclay（巴黎萨克雷大学）；Imperial College London（帝国理工学院）；TheraPanacea
During the last decades, the research community of medical imaging has witnessed continuous advances in image registration methods, which pushed the limits of the state-of-the-art and enabled the development of novel medical procedures. A particular type of image registration problem, known as slice-to-volume registration, played a fundamental role in areas like image guided surgeries and volumetric image reconstruction. However, to date, and despite the extensive literature available on this topic, no survey has been written to discuss this challenging problem. This paper introduces the first…
Medical Imaging Research Center, UZ Leuven（UZ Leuven医学成像研究中心）；Department of Electrical Engineering, ESAT/PSI, KU Leuven（KU Leuven电气工程系（ESAT/PSI））；Center for Craniofacial and Dental Genetics, Department of Oral and Craniofacial Sciences, University of Pittsburgh（匹兹堡大学口腔与颅面科学系颅面和牙科遗传学中心）；Department of Computing, Imperial College London（帝国理工学院计算机系）；IDSIA, USI Lugano（卢加诺大学IDSIA研究所）；Twitter（推特公司）；Department of Human Genetics, KU Leuven（KU Leuven人类遗传学系）
Registration is an essential tool in image analysis. Deep learning based alternatives have recently become popular, achieving competitive performance at a faster speed. However, many contemporary techniques are limited to volumetric representations, despite increased popularity of 3D surface and shape data in medical image analysis. We propose a one-step registration model for 3D surfaces that internalises a lower dimensional probabilistic deformation model (PDM) using conditional variational autoencoders (CVAE). The deformations are constrained to be diffeomorphic using an exponentiation laye…
Institute of Parallel and Distributed Systems, University of Stuttgart（斯图加特大学并行与分布式系统研究所）
Deformable image registration is a fundamental task in medical imaging. Due to the large computational complexity of deformable registration of volumetric images, conventional iterative methods usually face the tradeoff between the registration accuracy and the computation time in practice. In order to boost the registration performance in both accuracy and runtime, we propose a fast convolutional neural network. Specially, to efficiently utilize the memory resources and enlarge the model capacity, we adopt additive forwarding instead of channel concatenation and deepen the network in each enc…
Johns Hopkins Medical Institutes（约翰·霍普金斯医学院）；Johns Hopkins University（约翰·霍普金斯大学）；NVIDIA Corporation（英伟达公司）；Duke University Medical Center（杜克大学医学中心）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
In the last decade, convolutional neural networks (ConvNets) have been a major focus of research in medical image analysis. However, the performances of ConvNets may be limited by a lack of explicit consideration of the long-range spatial relationships in an image. Recently Vision Transformer architectures have been proposed to address the shortcomings of ConvNets and have produced state-of-the-art performances in many medical imaging applications. Transformers may be a strong candidate for image registration because their substantially larger receptive field enables a more precise comprehensi…
Volumetrically Consistent Implicit Atlas Learning via Neural Diffeomorphic Flow for Placenta MRI
MIT Computer Science and Artificial Intelligence Laboratory（麻省理工学院计算机科学与人工智能实验室）；Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院）；Boston Children’s Hospital, Harvard Medical School（哈佛医学院波士顿儿童医院）
Leibniz Universität Hannover（莱布尼茨汉诺威大学）；Hamburg University of Technology（汉堡工业大学）
Purpose: The facial recess is a delicate structure that must be protected in minimally invasive cochlear implant surgery. Current research estimates the drill trajectory by using endoscopy of the unique mastoid patterns. However, missing depth information limits available features for a registration to preoperative CT data. Therefore, this paper evaluates OCT for enhanced imaging of drill holes in mastoid bone and compares OCT data to original endoscopic images. Methods: A catheter-based OCT probe is inserted into a drill trajectory of a mastoid phantom in a translation-rotation manner to acqu…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins Medical Institutes（约翰斯·霍普金斯医学研究所）
In the last decade, convolutional neural networks (ConvNets) have dominated and achieved state-of-the-art performances in a variety of medical imaging applications. However, the performances of ConvNets are still limited by lacking the understanding of long-range spatial relations in an image. The recently proposed Vision Transformer (ViT) for image classification uses a purely self-attention-based model that learns long-range spatial relations to focus on the relevant parts of an image. Nevertheless, ViT emphasizes the low-resolution features because of the consecutive downsamplings, result i…
Differentiable rendering is a technique to connect 3D scenes with corresponding 2D images. Since it is differentiable, processes during image formation can be learned. Previous approaches to differentiable rendering focus on mesh-based representations of 3D scenes, which is inappropriate for medical applications where volumetric, voxelized models are used to represent anatomy. We propose a novel Projective Spatial Transformer module that generalizes spatial transformers to projective geometry, thus enabling differentiable volume rendering. We demonstrate the usefulness of this architecture on …
Aligning Fetal Anatomy with Kinematic Tree Log-Euclidean PolyRigid Transforms
Computer Science and Artificial Intelligence Lab, MIT, Cambridge, USA（麻省理工学院计算机科学与人工智能实验室）；California Institute of Technology, Pasadena, USA（加州理工学院）；Boston Children’s Hospital and Harvard Medical School, Boston, USA（哈佛医学院波士顿儿童医院）；The Hebrew University of Jerusalem, Jerusalem, Israel（耶路撒冷希伯来大学）
Advances in imaging and early cancer detection have increased interest in magnetic resonance (MR) guided focused ultrasound (MRgFUS) technologies for cancer treatment. MRgFUS ablation treatments could reduce surgical risks, preserve organ tissue/function, and improve patient quality of life. However, surgical resection and histological analysis remain the gold standard to assess cancer treatment response. For non-invasive ablation therapies such as MRgFUS, the treatment response must be determined through MR imaging biomarkers. However, current MR biomarkers are inconclusive and have not been …
University of Magdeburg（马格德堡大学）；University Hospital Magdeburg（马格德堡大学医院）
Recent studies demonstrated the eligibility of convolutional neural networks (CNNs) for solving the image registration problem. CNNs enable faster transformation estimation and greater generalization capability needed for better support during medical interventions. Conventional fully-supervised training requires a lot of high-quality ground truth data such as voxel-to-voxel transformations, which typically are attained in a too tedious and error-prone manner. In our work, we use weakly-supervised learning, which optimizes the model indirectly only via segmentation masks that are a more access…
In medical image segmentation, supervised deep networks' success comes at the cost of requiring abundant labeled data. While asking domain experts to annotate only one or a few of the cohort's images is feasible, annotating all available images is impractical. This issue is further exacerbated when pre-trained deep networks are exposed to a new image dataset from an unfamiliar distribution. Using available open-source data for ad-hoc transfer learning or hand-tuned techniques for data augmentation only provides suboptimal solutions. Motivated by atlas-based segmentation, we propose a novel vol…
CMRINet: Joint Groupwise Registration and Segmentation for Cardiac Function Quantification from Cine-MRI
Leiden University Medical Center（莱顿大学医学中心）；University of Sheffield（谢菲尔德大学）；INSIGNEO；Institute for In Silico Medicine（计算医学研究所）；Department of Infection, Immunity and Cardiovascular Disease（感染、免疫与心血管疾病部门）；Sheffield Teaching Hospitals（谢菲尔德教学医院）；Department of Clinical Radiology（临床放射学部门）
University of Alberta（阿尔伯塔大学）；Mazankowski Alberta Heart Institute（马赞科夫斯基阿尔伯塔心脏研究所）
Purpose: Echocardiography is commonly used as a non-invasive imaging tool in clinical practice for the assessment of cardiac function. However, delineation of the left ventricle is challenging due to the inherent properties of ultrasound imaging, such as the presence of speckle noise and the low signal-to-noise ratio. Methods: We propose a semi-automated segmentation algorithm for the delineation of the left ventricle in temporal 3D echocardiography sequences. The method requires minimal user interaction and relies on a diffeomorphic registration approach. Advantages of the method include no d…
Multi-atlas segmentation (MAS) is a popular image segmentation technique for medical images. In this work, we improve the performance of MAS by correcting registration errors before label fusion. Specifically, we use a volumetric displacement field to refine registrations based on image anatomical appearance and predicted labels. We show the influence of the initial spatial alignment as well as the beneficial effect of using label information for MAS performance. Experiments demonstrate that the proposed refinement approach improves MAS performance on a 3D magnetic resonance dataset of the kne…
The detailed reconstruction of neural anatomy for connectomics studies requires a combination of resolution and large three-dimensional data capture provided by serial section electron microscopy (ssEM). The convergence of high throughput ssEM imaging and improved tissue preparation methods now allows ssEM capture of complete specimen volumes up to cubic millimeter scale. The resulting multi-terabyte image sets span thousands of serial sections and must be precisely registered into coherent volumetric forms in which neural circuits can be traced and segmented. This paper introduces a Signal Wh…
Enhancing and Accelerating Brain MRI through Deep Learning Reconstruction Using Prior Subject-Specific Imaging
Department of Electrical and Software Engineering, University of Calgary（电气与软件工程系，卡尔加里大学）；Departments of Radiology and Clinical Neuroscience, University of Calgary（放射学与临床神经科学系，卡尔加里大学）；Department of Biomedical Engineering, University of Calgary（生物医学工程系，卡尔加里大学）；Department of Radiology, University Hospital Basel（放射学系，巴塞尔大学医院）；Department of Neurology, University Hospital Basel（神经病学系，巴塞尔大学医院）；Hotchkiss Brain Institute, University of Calgary（霍奇基斯脑研究所，卡尔加里大学）
ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究中心）
Image registration of liver dynamic contrast-enhanced computed tomography (DCE-CT) is crucial for diagnosis and image-guided surgical planning of liver cancer. However, intensity variations due to the flow of contrast agents combined with complex spatial motion induced by respiration brings great challenge to existing intensity-based registration methods. To address these problems, we propose a novel structure-aware registration method by incorporating structural information of related organs with segmentation-guided deep registration network. Existing segmentation-guided registration methods …
Anatomically plausible image registration often requires volumetric preservation. Previous approaches to incompressible image registration have exploited relaxed constraints, ad hoc optimisation methods or practically intractable computational schemes. Divergence-free velocity fields have been used to achieve incompressibility in the continuous domain, although, after discretisation, no guarantees have been provided. In this paper, we introduce stationary velocity fields (SVFs) parameterised by divergence-conforming B-splines in the context of image registration. We demonstrate that sparse lin…
Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis
Department of Radiation Oncology, TUM University Hospital, Munich, Germany（放射肿瘤科，慕尼黑技术大学医院，德国）；Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM)（人工智能在医疗和健康领域的主任，慕尼黑技术大学）；Chair for AI for Image-Guided Diagnosis and Therapy, Technical University of Munich (TUM)（人工智能在影像引导诊断和治疗领域的主任，慕尼黑技术大学）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML），德国慕尼黑）；Department of Computing, Imperial College London, London, UK（计算系，伦敦帝国学院，英国伦敦）；Deutsches Konsortium für Translationale Krebsforschung (DKTK), Partner Site Munich, Munich, Germany（德国转化癌症研究联盟（DKTK），慕尼黑分部，德国慕尼黑）
Mamba-driven MRI-to-CT Synthesis for MRI-only Radiotherapy Planning
School of Electrical；Computer Engineering, National Technical University of Athens Archimedes, Athena Research Center CentraleSup\'elec, University of Paris-Saclay
Ecole de Technologie Superieure（高等技术学院）；McGill University（麦吉尔大学）
The contributions of this thesis stem from technology developed to analyse large sets of volumetric images in terms of atom-like features extracted in 3D image space, following SIFT algorithm in 2D image space. New feature properties are introduced including a binary feature sign, analogous to an electrical charge, and a discrete set of symmetric feature orientation states in 3D space. These new properties are leveraged to extend feature invariance to include the sign inversion and parity (SP) transform, analogous to the charge conjugation and parity (CP) transform between a particle and its a…
University of Oxford（牛津大学）；University College London（伦敦大学学院）；InstaDeep（InstaDeep公司）；City University of Hong Kong（香港城市大学）；Radboud University Nijmegen Medical Centre（拉德堡德大学奈梅亨医学中心）
The ability to adapt medical image segmentation networks for a novel class such as an unseen anatomical or pathological structure, when only a few labelled examples of this class are available from local healthcare providers, is sought-after. This potentially addresses two widely recognised limitations in deploying modern deep learning models to clinical practice, expertise-and-labour-intensive labelling and cross-institution generalisation. This work presents the first 3D few-shot interclass segmentation network for medical images, using a labelled multi-institution dataset from prostate canc…
Investigating the impact of kernel harmonization and deformable registration on inspiratory and expiratory chest CT images for people with COPD
Vanderbilt University（范德堡大学）；Insitro；Vanderbilt University Medical Center（范德堡大学医学中心）；Department of Veterans Affairs Medical Center（退伍军人事务部医疗中心）；Vanderbilt University Institute of Imaging Science（范德堡大学成像科学研究所）
针对 COPD 患者吸呼气 CT 因重建核差异导致定量分析误差的问题，提出两阶段流程：先用循环 GAN 进行核 harmonization，再进行可变形配准，显著降低肺气肿测量不一致性并提高配准精度。
Volumetric Reconstruction of Prostatectomy Specimens from Histology
Charité – Universitätsmedizin Berlin（柏林夏里特医学院）；Freie Universität Berlin（柏林自由大学）；Humboldt Universität zu Berlin（柏林洪堡大学）；MVZ Helios Klinikum Emil von Behring（赫利俄斯埃米尔·冯·贝林诊所MVZ）；University Hospital RWTH Aachen（亚琛工业大学医院）
