针对 OCT 散斑噪声中 A 扫描体素相关导致自监督去噪失效的问题,提出融合逐切片训练与配准模块的端到端框架,实验证明其优于现有自监督模型。
摘要:
光学相干断层扫描(OCT)成像固有强烈的散斑噪声,这对视网膜结构的准确定量分析构成重大障碍,而该分析是推进临床诊断和疾病监测的关键。用于保持结构的降噪的基于学习的自监督方法已表现出优于传统方法的性能,但在 OCT 成像中面临独特挑战。由相干 A 扫描光束产生的体素具有高度相关性,这违背了像素噪声独立的假设,从而削弱了自监督学习方法的效果。我们开展实验,证明了现有模型因这一独立性假设而存在的局限。随后,我们提出一种专为 OCT 图像去噪定制的新型端到端自监督学习框架,将逐切片训练和配准模块集成到一个网络中。针对所提方法进行了大量消融研究。与此前已发表的自监督去噪模型相比,所提框架表现出更优性能,有望作为预处理步骤,以实现更好的分割性能和定量分析。
英文摘要:
Strong speckle noise is inherent to optical coherence tomography (OCT) imaging and represents a significant obstacle for accurate quantitative analysis of retinal structures which is key for advances in clinical diagnosis and monitoring of disease. Learning-based self-supervised methods for structure-preserving noise reduction have demonstrated superior performance over traditional methods but face unique challenges in OCT imaging. The high correlation of voxels generated by coherent A-scan beams undermines the efficacy of self-supervised learning methods as it violates the assumption of independent pixel noise. We conduct experiments demonstrating limitations of existing models due to this independence assumption. We then introduce a new end-to-end self-supervised learning framework specifically tailored for OCT image denoising, integrating slice-by-slice training and registration modules into one network. An extensive ablation study is conducted for the proposed approach. Comparison to previously published self-supervised denoising models demonstrates improved performance of the proposed framework, potentially serving as a preprocessing step towards superior segmentation performance and quantitative analysis.
Zhejiang University(浙江大学);Shandong Medical Imaging Research Institute(山东医学影像学研究所);Cheeloo College of Medicine, Shandong University(山东大学齐鲁医学院);Shanghai Jiao Tong University(上海交通大学)
Multi-slice magnetic resonance images of the fetal brain are usually contaminated by severe and arbitrary fetal and maternal motion. Hence, stable and robust motion correction is necessary to reconstruct high-resolution 3D fetal brain volume for clinical diagnosis and quantitative analysis. However, the conventional registration-based correction has a limited capture range and is insufficient for detecting relatively large motions. Here, we present a novel Affinity Fusion-based Framework for Iteratively Random Motion (AFFIRM) correction of the multi-slice fetal brain MRI. It learns the sequent…
Ryerson University(瑞尔森大学);University of Toronto(多伦多大学);Institute for Biomedical Engineering, Science and Technology (iBEST)(生物医学工程、科学与技术研究所(iBEST))
We propose FlowReg, a deep learning-based framework for unsupervised image registration for neuroimaging applications. The system is composed of two architectures that are trained sequentially: FlowReg-A which affinely corrects for gross differences between moving and fixed volumes in 3D followed by FlowReg-O which performs pixel-wise deformations on a slice-by-slice basis for fine tuning in 2D. The affine network regresses the 3D affine matrix based on a correlation loss function that enforces global similarity. The deformable network operates on 2D image slices based on the optical flow netw…
Rigid slice-to-volume registration is a challenging task, which finds application in medical imaging problems like image fusion for image guided surgeries and motion correction for volume reconstruction. It is usually formulated as an optimization problem and solved using standard continuous methods. In this paper, we discuss how this task be formulated as a discrete labeling problem on a graph. Inspired by previous works on discrete estimation of linear transformations using Markov Random Fields (MRFs), we model it using a pairwise MRF, where the nodes are associated to the rigid parameters, …
Pattern Recognition Lab, FAU Erlangen-Nürnberg(埃尔朗根-纽伦堡大学模式识别实验室);Department of Ophthalmology, Universitätsklinikum Erlangen(埃尔朗根大学医院眼科)
Ophthalmological imaging utilizes different imaging systems, such as color fundus, infrared, fluorescein angiography, optical coherence tomography (OCT) or OCT angiography. Multiple images with different modalities or acquisition times are often analyzed for the diagnosis of retinal diseases. Automatically aligning the vessel structures in the images by means of multi-modal registration can support the ophthalmologists in their work. Our method uses a convolutional neural network to extract features of the vessel structure in multi-modal retinal images. We jointly train a keypoint detection an…
We propose a method to predict severity of age related macular degeneration (AMD) from input optical coherence tomography (OCT) images. Although there is no standard clinical severity scale for AMD, we leverage deep learning (DL) based image registration and clustering methods to identify diseased cases and predict their severity. Experiments demonstrate our approach's disease classification performance matches state of the art methods. The predicted disease severity performs well on previously unseen data. Registration output provides better explainability than class activation maps regarding…
Institute of Medical Informatics, Universität zu Lübeck(吕贝克大学医学信息学研究所)
Multimodal image registration is a very challenging problem for deep learning approaches. Most current work focuses on either supervised learning that requires labelled training scans and may yield models that bias towards annotated structures or unsupervised approaches that are based on hand-crafted similarity metrics and may therefore not outperform their classical non-trained counterparts. We believe that unsupervised domain adaptation can be beneficial in overcoming the current limitations for multimodal registration, where good metrics are hard to define. Domain adaptation has so far been…
University of Sao Paulo(圣保罗大学);Faculty of Philosophy, Science and Languages of Ribeirao Preto(里贝朗普雷图哲学、科学与语言学院);Department of Physics Applied to Medicine and Biology(医学与生物学应用物理系)
Mutual information (MI) is the standard method used in image registration and the most studied one but can diverge and produce wrong results when used in an automated manner. In this study we compared the results of the ITK Mattes MI function, used in 3D Slicer and ITK derived software solutions, and our own MICUDA Shannon and Tsallis MI functions under the translation, rotation and scale transforms in a 3D mathematical space. This comparison allows to understand why registration fails in some circumstances and how to produce a more robust automated algorithm to register medical images. Since …
In ophthalmological imaging, multiple imaging systems, such as color fundus, infrared, fluorescein angiography, optical coherence tomography (OCT) or OCT angiography, are often involved to make a diagnosis of retinal disease. Multi-modal retinal registration techniques can assist ophthalmologists by providing a pixel-based comparison of aligned vessel structures in images from different modalities or acquisition times. To this end, we propose an end-to-end trainable deep learning method for multi-modal retinal image registration. Our method extracts convolutional features from the vessel struc…
A Hybrid Deep Feature-Based Deformable Image Registration Method for Pathology Images
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);School of Mathematics, Sun Yat-sen University(中山大学数学学院);Stanford University(斯坦福大学);Guangdong Medical University(广东医科大学);Xiaohe Healthcare ByteDance(字节跳动小河医疗)
This report presents a generic image registration benchmark with automatic evaluation using landmark annotations. The key features of the BIRL framework are: easily extendable, performance evaluation, parallel experimentation, simple visualisations, experiment's time-out limit, resuming unfinished experiments. From the research practice, we identified and focused on these two main use-cases: (a) comparison of user's (newly developed) method with some State-of-the-Art (SOTA) methods on a common dataset and (b) experimenting SOTA methods on user's custom dataset (which should contain landmark an…
This letter proposes a synthetic aperture radar (SAR) image registration method named Feature-Area Optimization (FAO). First, the traditional area-based optimization model is reconstructed and decomposed into three key but uncertain factors: initialization, slice set and regularization. Next, structural features are extracted by scale invariant feature transform (SIFT) in dual-resolution space (SIFT-DRS), a novel SIFT-Like method dedicated to FAO. Then, the three key factors are determined based on these features. Finally, solving the factor-determined optimization model can get the registrati…
This paper aims to solve a fundamental problem in intensity-based 2D/3D registration, which concerns the limited capture range and need for very good initialization of state-of-the-art image registration methods. We propose a regression approach that learns to predict rotation and translations of arbitrary 2D image slices from 3D volumes, with respect to a learned canonical atlas co-ordinate system. To this end, we utilize Convolutional Neural Networks (CNNs) to learn the highly complex regression function that maps 2D image slices into their correct position and orientation in 3D space. Our a…
MIT(麻省理工学院);Boston Children’s Hospital(波士顿儿童医院);Harvard Medical School(哈佛医学院);University College London(伦敦大学学院);Massachusetts General Hospital(麻省总医院)
Volumetric reconstruction of fetal brains from multiple stacks of MR slices, acquired in the presence of almost unpredictable and often severe subject motion, is a challenging task that is highly sensitive to the initialization of slice-to-volume transformations. We propose a novel slice-to-volume registration method using Transformers trained on synthetically transformed data, which model multiple stacks of MR slices as a sequence. With the attention mechanism, our model automatically detects the relevance between slices and predicts the transformation of one slice using information from othe…
CEA, Fundamental Research Division (DRF), Institute Jacob, Molecular Imaging Research Center (MIRCen)(法国原子能与可替代能源委员会(CEA)基础研究部雅各布研究所分子影像研究中心(MIRCen));WITSEE(WITSEE 公司)
A significant challenge for brain histological data analysis is to precisely identify anatomical regions in order to perform accurate local quantifications and evaluate therapeutic solutions. Usually, this task is performed manually, becoming therefore tedious and subjective. Another option is to use automatic or semi-automatic methods, among which segmentation using digital atlases co-registration. However, most available atlases are 3D, whereas digitized histological data are 2D. Methods to perform such 2D-3D segmentation from an atlas are required. This paper proposes a strategy to automati…
Institute of Medical Technology and Intelligent Sys.(医学技术与智能系统研究所);Hamburg University of Technology(汉堡理工大学);Faculty of Electrical and Electronics Eng.(电气与电子工程系);Kaunas University of Technology(卡斯图技术大学);Interdisciplinary Competence Center for Interface Research(界面研究跨学科能力中心);SustAInLivWork Center of Excellence(SustAInLivWork卓越研究中心)
University of Edinburgh(爱丁堡大学);Imperial College London(帝国理工学院);University of Essex(埃塞克斯大学)
Magnetic Resonance Imaging (MRI) typically recruits multiple sequences (defined here as "modalities"). As each modality is designed to offer different anatomical and functional clinical information, there are evident disparities in the imaging content across modalities. Inter- and intra-modality affine and non-rigid image registration is an essential medical image analysis process in clinical imaging, as for example before imaging biomarkers need to be derived and clinically evaluated across different MRI modalities, time phases and slices. Although commonly needed in real clinical scenarios, …
IMITATE: Image Registration with Context for unknown time frame recovery
Centre de Recherches en Cancérologie de Toulouse, INSERM UMR1037, Team RADOPT(图卢兹癌症研究中心,INSERM UMR1037,RADOPT团队);Université de Toulouse(图卢兹大学);Institut Universitaire du Cancer – Oncopole Claudius Régaud(癌症大学研究所 – 克劳迪乌斯·雷加德肿瘤中心);Institut de Mathématiques de Toulouse (UMR 5219), CNRS, Université de Toulouse(图卢兹数学研究所(UMR 5219),CNRS,图卢兹大学)
Full end-to-end diagnostic workflow automation of 3D OCT via foundation model-driven AI for retinal diseases
Sun Yat-sen University(中山大学);Southern University of Science and Technology(南方科技大学);The University of Hong Kong(香港大学);The Hong Kong Polytechnic University(香港理工大学);Zhongshan City People's Hospital(中山市人民医院);Guangdong Medical University(广东医科大学);The Third Affiliated Hospital of Sun Yat-sen University(中山大学附属第三医院);Zhaoqing Gaoyao People’s Hospital(肇庆高要人民医院);Tianjin Medical University Eye Hospital(天津医科大学眼科医院);Capital Medical University(首都医科大学);Beijing Tongren Hospital(北京同仁医院)
Beijing Institute of Technology(北京理工大学);Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences(中国科学院宁波材料技术与工程研究所);Cixi Institute of Biomedical Engineering(慈溪生物医学工程研究所);Ningbo University of Technology(宁波工程学院)
Retinal Optical Coherence Tomography Angiography (OCTA) with high-resolution is important for the quantification and analysis of retinal vasculature. However, the resolution of OCTA images is inversely proportional to the field of view at the same sampling frequency, which is not conducive to clinicians for analyzing larger vascular areas. In this paper, we propose a novel Sparse-based domain Adaptation Super-Resolution network (SASR) for the reconstruction of realistic 6x6 mm2/low-resolution (LR) OCTA images to high-resolution (HR) representations. To be more specific, we first perform a simp…
School of Biomedical Engineering and Imaging Sciences, King’s College London(伦敦国王学院生物医学工程与成像科学学院);Department of Women and Children’s Health, School of Life Course Sciences, King’s College London(伦敦国王学院生命历程科学学院妇幼健康系)
Accurately estimating and correcting the motion artifacts are crucial for 3D image reconstruction of the abdominal and in-utero magnetic resonance imaging (MRI). The state-of-art methods are based on slice-to-volume registration (SVR) where multiple 2D image stacks are acquired in three orthogonal orientations. In this work, we present a novel reconstruction pipeline that only needs one orientation of 2D MRI scans and can reconstruct the full high-resolution image without masking or registration steps. The framework consists of two main stages: the respiratory motion estimation using a self-su…
University College London(伦敦大学学院);UCL Institute of Neurology(伦敦大学学院神经病学研究所);Wellcome / EPSRC Centre for Interventional and Surgical Sciences, UCL(伦敦大学学院威康/EPSRC介入与外科科学中心)
Image registration between histology and magnetic resonance imaging (MRI) is a challenging task due to differences in structural content and contrast. Too thick and wide specimens cannot be processed all at once and must be cut into smaller pieces. This dramatically increases the complexity of the problem, since each piece should be individually and manually pre-aligned. To the best of our knowledge, no automatic method can reliably locate such piece of tissue within its respective whole in the MRI slice, and align it without any prior information. We propose here a novel automatic approach to…
Hunan University of Technology and Business(湖南工商大学);Utah State University(犹他州立大学);University of Idaho(爱达荷大学)
Liver segmentation from abdominal CT images is an essential step for liver cancer computer-aided diagnosis and surgical planning. However, both the accuracy and robustness of existing liver segmentation methods cannot meet the requirements of clinical applications. In particular, for the common clinical cases where the liver tissue contains major pathology, current segmentation methods show poor performance. In this paper, we propose a novel low-rank tensor decomposition (LRTD) based multi-atlas segmentation (MAS) framework that achieves accurate and robust pathological liver segmentation of C…
SCoPE-Reg: Efficient Rigid Ultrasound Slice-to-Volume Registration via State-Space Correlation and Closed-Form Pose Estimation
Institute for AI in Medicine (IKIM), University Hospital Essen(埃森大学医院医学人工智能研究所(IKIM));RACOON Study Group, Site Essen(RACOON研究组埃森站点);Department of Physics, Technical University Dortmund(多特蒙德工业大学物理系);German Cancer Consortium (DKTK), Partner Site Essen(德国癌症联盟(DKTK)埃森合作站点);Medical Faculty and Faculty of Computer Science, University of Duisburg-Essen(杜伊斯堡-埃森大学医学院与计算机科学学院)
Medical image slice interpolation is an active field of research. The methods for this task can be categorized into two broad groups: intensity-based and object-based interpolation methods. While intensity-based methods are generally easier to perform and less computationally expensive, object-based methods are capable of producing more accurate results and account for deformable changes in the objects within the slices. In this paper, performance of two well-known object-based interpolation methods is analyzed and compared. Here, a deformable registration-based method specifically designed fo…
A novel framework for fully-automated co-registration of intravascular ultrasound and optical coherence tomography imaging data
Barts Health NHS Trust(巴茨健康国民医疗服务信托);Huazhong University of Science and Technology(华中科技大学);Queen Mary, University of London(伦敦玛丽女王大学);Queen Mary University London(伦敦玛丽女王大学);University College London(伦敦大学学院);Leiden University Medical Center(莱顿大学医学中心);University of Bern(伯尔尼大学)
University of Auckland(奥克兰大学);Queen Mary University of London(伦敦玛丽女王大学);University of Oxford(牛津大学)
Purpose: To demonstrate the feasibility and performance of a fully automated deep learning framework to estimate myocardial strain from short-axis cardiac magnetic resonance tagged images. Methods and Materials: In this retrospective cross-sectional study, 4508 cases from the UK Biobank were split randomly into 3244 training and 812 validation cases, and 452 test cases. Ground truth myocardial landmarks were defined and tracked by manual initialization and correction of deformable image registration using previously validated software with five readers. The fully automatic framework consisted …
In in-utero MRI, motion correction for fetal body and placenta poses a particular challenge due to the presence of local non-rigid transformations of organs caused by bending and stretching. The existing slice-to-volume registration (SVR) reconstruction methods are widely employed for motion correction of fetal brain that undergoes only rigid transformation. However, for reconstruction of fetal body and placenta, rigid registration cannot resolve the issue of misregistrations due to deformable motion, resulting in degradation of features in the reconstructed volume. We propose a Deformable SVR…
Variously stained histology slices are routinely used by pathologists to assess extracted tissue samples from various anatomical sites and determine the presence or extent of a disease. Evaluation of sequential slides is expected to enable a better understanding of the spatial arrangement and growth patterns of cells and vessels. In this paper we present a practical two-step approach based on diffeomorphic registration to align digitized sequential histopathology stained slides to each other, starting with an initial affine step followed by the estimation of a detailed deformation field.
Magnetic resonance imaging (MRI) has great potential to improve prostate cancer diagnosis. It can spare men with a normal exam from undergoing invasive biopsy while making biopsies more accurate in men with lesions suspicious for cancer. Yet, the subtle differences between cancer and confounding conditions, render the interpretation of MRI challenging. The tissue collected from patients that undergo pre-surgical MRI and radical prostatectomy provides a unique opportunity to correlate histopathology images of the entire prostate with MRI in order to accurately map the extent of prostate cancer …
Rensselaer Polytechnic Institute(伦斯勒理工学院);National Institutes of Health(美国国立卫生研究院)
Fusing intra-operative 2D transrectal ultrasound (TRUS) image with pre-operative 3D magnetic resonance (MR) volume to guide prostate biopsy can significantly increase the yield. However, such a multimodal 2D/3D registration problem is a very challenging task. In this paper, we propose an end-to-end frame-to-volume registration network (FVR-Net), which can efficiently bridge the previous research gaps by aligning a 2D TRUS frame with a 3D TRUS volume without requiring hardware tracking. The proposed FVR-Net utilizes a dual-branch feature extraction module to extract the information from TRUS fr…
Universidad Politécnica de Madrid(马德里理工大学);Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN)(生物医学研究网络中心(生物工程、生物材料与纳米医学));King’s College London(伦敦国王学院)
Magnetic resonance imaging of whole fetal body and placenta is limited by different sources of motion affecting the womb. Usual scanning techniques employ single-shot multi-slice sequences where anatomical information in different slices may be subject to different deformations, contrast variations or artifacts. Volumetric reconstruction formulations have been proposed to correct for these factors, but they must accommodate a non-homogeneous and non-isotropic sampling, so regularization becomes necessary. Thus, in this paper we propose a deep generative prior for robust volumetric reconstructi…
Imaging Informatics Division(成像信息学部门);Bioinformatics Institute(生物信息学研究所);A*STAR(新加坡科技研究局);Lucence Diagnostics(Lucence诊断公司);Institute of Bioengineering and Nanotechnology(生物工程与纳米技术研究所);Singapore General Hospital(新加坡综合医院);National Cancer Centre Singapore(新加坡国家癌症中心);Institute of Molecular and Cell Biology(分子与细胞生物学研究所);National University of Singapore(新加坡国立大学);Singapore Eye Research Institute(新加坡眼科研究所);Image and Pervasive Access Lab(图像与普适访问实验室)
Motivation: High resolution 2D whole slide imaging provides rich information about the tissue structure. This information can be a lot richer if these 2D images can be stacked into a 3D tissue volume. A 3D analysis, however, requires accurate reconstruction of the tissue volume from the 2D image stack. This task is not trivial due to the distortions that each individual tissue slice experiences while cutting and mounting the tissue on the glass slide. Performing registration for the whole tissue slices may be adversely affected by the deformed tissue regions. Consequently, regional registratio…
Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences(中国科学院宁波材料技术与工程研究所慈溪生物医学工程研究所);University of Chinese Academy of Sciences(中国科学院大学);Baidu Inc(百度公司);Southern University of Science and Technology(南方科技大学);Zhongshan Ophthalmic Center, Sun Yat-sen University(中山大学中山眼科中心);Inception Institute of Artificial Intelligence(起源人工智能研究院)
Precise characterization and analysis of iris shape from Anterior Segment OCT (AS-OCT) are of great importance in facilitating diagnosis of angle-closure-related diseases. Existing methods focus solely on analyzing structural properties identified from the 2D slice, while accurate characterization of morphological changes of iris shape in 3D AS-OCT may be able to reveal in addition the risk of disease progression. In this paper, we propose a novel framework for reconstruction and quantification of 3D iris surface from AS-OCT imagery. We consider it to be the first work to detect angle-closure …
University of Sussex(萨塞克斯大学);Polytechnique Montreal(蒙特利尔理工学院);Cardiff University(卡迪夫大学);University College London(伦敦大学学院);Harvard Medical School(哈佛医学院);Massachusetts Institute of Technology(麻省理工学院)
Nonlinear inter-modality registration is often challenging due to the lack of objective functions that are good proxies for alignment. Here we propose a synthesis-by-registration method to convert this problem into an easier intra-modality task. We introduce a registration loss for weakly supervised image translation between domains that does not require perfectly aligned training data. This loss capitalises on a registration U-Net with frozen weights, to drive a synthesis CNN towards the desired translation. We complement this loss with a structure preserving constraint based on contrastive l…
University College London(伦敦大学学院);Inria(法国国家信息与自动化研究所);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Massachusetts Institute of Technology(麻省理工学院)
Joint registration of a stack of 2D histological sections to recover 3D structure (``3D histology reconstruction'') finds application in areas such as atlas building and validation of \emph{in vivo} imaging. Straightforward pairwise registration of neighbouring sections yields smooth reconstructions but has well-known problems such as ``banana effect'' (straightening of curved structures) and ``z-shift'' (drift). While these problems can be alleviated with an external, linearly aligned reference (e.g., Magnetic Resonance (MR) images), registration is often inaccurate due to contrast difference…
University of Oxford(牛津大学);University College London(伦敦大学学院);InstaDeep(InstaDeep公司);City University of Hong Kong(香港城市大学);Radboud University Nijmegen Medical Centre(拉德堡德大学奈梅亨医学中心)
The ability to adapt medical image segmentation networks for a novel class such as an unseen anatomical or pathological structure, when only a few labelled examples of this class are available from local healthcare providers, is sought-after. This potentially addresses two widely recognised limitations in deploying modern deep learning models to clinical practice, expertise-and-labour-intensive labelling and cross-institution generalisation. This work presents the first 3D few-shot interclass segmentation network for medical images, using a labelled multi-institution dataset from prostate canc…
Manual preparation of tissue slices for microscopy imaging can introduce tissue tears and overlaps. Typically, further digital processing algorithms such as registration and 3D reconstruction from tissue image stacks cannot handle images with tissue tear/overlap artifacts, and so such images are usually discarded. In this paper, we propose an imaging model and an algorithm to digitally separate overlapping tissue data of mouse brain images into two layers. We show the correctness of our model and the algorithm by comparing our results with the ground truth.
Medical University of Vienna(维也纳医科大学);Centre for Eye Research Australia(澳大利亚眼科研究中心);Royal Victorian Eye and Ear Hospital(皇家维多利亚眼耳医院);The University of Melbourne(墨尔本大学)
Longitudinal imaging is capable of capturing the static ana\-to\-mi\-cal structures and the dynamic changes of the morphology resulting from aging or disease progression. Self-supervised learning allows to learn new representation from available large unlabelled data without any expert knowledge. We propose a deep learning self-supervised approach to model disease progression from longitudinal retinal optical coherence tomography (OCT). Our self-supervised model takes benefit from a generic time-related task, by learning to estimate the time interval between pairs of scans acquired from the sa…