Registration of serial sections: An evaluation method based on distortions of the ground truths
Hannover Medical School（汉诺威医学院）；Leibniz-Fachhochschule School of Business（莱布尼茨应用科学大学商学院）；Nara Institute of Science and Technology（奈良先端科学技术大学院大学）；Biomedical Research in Endstage and Obstructive Lung Disease Hannover (BREATH), Member of the German Center for Lung Research (DZL)（汉诺威终末期与阻塞性肺病生物医学研究（BREATH），德国肺研究中心（DZL）成员）
Oleg Lobachev, Takuya Funatomi, Alexander Pfaffenroth, Reinhold Förster, Lars Knudsen, Christoph Wrede, Michael Guthe, David Haberthür, Ruslan Hlushchuk, Thomas… 展开作者
Oleg Lobachev, Takuya Funatomi, Alexander Pfaffenroth, Reinhold Förster, Lars Knudsen, Christoph Wrede, Michael Guthe, David Haberthür, Ruslan Hlushchuk, Thomas Salaets, Jaan Toelen, Simone Gaffling, Christian Mühlfeld, Roman Grothausmann
Registration of histological serial sections is a challenging task. Serial sections exhibit distortions and damage from sectioning. Missing information on how the tissue looked before cutting makes a realistic validation of 2D registrations extremely difficult.
This work proposes methods for ground-truth-based evaluation of registrations. Firstly, we present a methodology to generate test data for registrations. We distort an innately registered image stack in the manner similar to the cutting distortion of serial sections. Test cases are generated from existing 3D data sets, thus the ground truth is known. Secondly, our test case generation premises evaluation of the registrations with known ground truths. Our methodology for such an evaluation technique distinguishes this work from other approaches. Both under- and over-registration become evident in our evaluations. We also survey existing validation efforts.
We present a full-series evaluation across six different registration methods applied to our distorted 3D data sets of animal lungs. Our distorted and ground truth data sets are made publicly available.
Ulsan National Institute of Science and Technology（蔚山科学技术院）；The Rockefeller University（洛克菲勒大学）；Harvard Medical School（哈佛医学院）
The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for …
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）
In this paper, we propose a novel non-iterative algorithm to simultaneously estimate optimal rigid transformation for serial section images, which is a key component in volume reconstruction of serial sections of biological tissue. In order to avoid error accumulation and propagation caused by current algorithms, we add extra condition that the position of the first and the last section images should remain unchanged. This constrained simultaneous registration problem has not been solved before. Our algorithm method is non-iterative, it can simultaneously compute rigid transformation for a lar…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；Temple University（天普大学）
There has been a long pursuit for precise and reproducible glomerular quantification on renal pathology to leverage both research and practice. When digitizing the biopsy tissue samples using whole slide imaging (WSI), a set of serial sections from the same tissue can be acquired as a stack of images, similar to frames in a video. In radiology, the stack of images (e.g., computed tomography) are naturally used to provide 3D context for organs, tissues, and tumors. In pathology, it is appealing to do a similar 3D assessment. However, the 3D identification and association of large-scale glomerul…
STAR: A Fast and Robust Rigid Registration Framework for Serial Histopathological Images
School of Biological Science and Medical Engineering, Beihang University, Beijing, China（生物科学与医学工程学院，北航，北京，中国）；School of Computer Science and Engineering, Beihang University, Beijing, China（计算机科学与工程学院，北航，北京，中国）
Imperial College London（帝国理工学院）；University of Manchester（曼彻斯特大学）
We present a generic method for assessing the quality of non-rigid registration (NRR) algorithms, that does not depend on the existence of any ground truth, but depends solely on the data itself. The data is a set of images. The output of any NRR of such a set of images is a dense correspondence across the whole set. Given such a dense correspondence, it is possible to build various generative statistical models of appearance variation across the set. We show that evaluating the quality of the registration can be mapped to the problem of evaluating the quality of the resultant statistical mode…
The detailed reconstruction of neural anatomy for connectomics studies requires a combination of resolution and large three-dimensional data capture provided by serial section electron microscopy (ssEM). The convergence of high throughput ssEM imaging and improved tissue preparation methods now allows ssEM capture of complete specimen volumes up to cubic millimeter scale. The resulting multi-terabyte image sets span thousands of serial sections and must be precisely registered into coherent volumetric forms in which neural circuits can be traced and segmented. This paper introduces a Signal Wh…
Seoul National University（首尔国立大学）；Soongsil University（崇实大学）
Accurate three-dimensional perception is a fundamental task in several computer vision applications. Recently, commercial RGB-depth (RGB-D) cameras have been widely adopted as single-view depth-sensing devices owing to their efficient depth-sensing abilities. However, the depth quality of most RGB-D sensors remains insufficient owing to the inherent noise from a single-view environment. Recently, several studies have focused on the single-view depth enhancement of RGB-D cameras. Recent research has proposed deep-learning-based approaches that typically train networks using high-quality supervi…
Yale University（耶鲁大学）；Yale School of Medicine（耶鲁大学医学院）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；Chinese Academy of Sciences（中国科学院）
Multimodal image registration has many applications in diagnostic medical imaging and image-guided interventions, such as Transcatheter Arterial Chemoembolization (TACE) of liver cancer guided by intraprocedural CBCT and pre-operative MR. The ability to register peri-procedurally acquired diagnostic images into the intraprocedural environment can potentially improve the intra-procedural tumor targeting, which will significantly improve therapeutic outcomes. However, the intra-procedural CBCT often suffers from suboptimal image quality due to lack of signal calibration for Hounsfield unit, limi…
AI-assisted surgeries have drawn the attention of the medical image research community due to their real-world impact on improving surgery success rates. For image-guided surgeries, such as Cochlear Implants (CIs), accurate object segmentation can provide useful information for surgeons before an operation. Recently published image segmentation methods that leverage machine learning usually rely on a large number of manually predefined ground truth labels. However, it is a laborious and time-consuming task to prepare the dataset. This paper presents a novel technique using a self-supervised 3D…
Aachen Institute for Advanced Study in Computational Engineering Science (AICES), RWTH Aachen University（亚琛工业大学计算工程科学高级研究院（AICES））；Peter L. Reichertz Institute for Medical Informatics, University of Braunschweig – Institute of Technology and Hannover Medical School（布伦瑞克工业大学与汉诺威医学院彼得·L·赖歇茨医学信息学研究所）
Region of interest (ROI) alignment in medical images plays a crucial role in diagnostics, procedure planning, treatment, and follow-up. Frequently, a model is represented as triangulated mesh while the patient data is provided from CAT scanners as pixel or voxel data. Previously, we presented a 2D method for curve-to-pixel registration. This paper contributes (i) a general mesh-to-raster (M2R) framework to register ROIs in multi-modal images; (ii) a 3D surface-to-voxel application, and (iii) a comprehensive quantitative evaluation in 2D using ground truth provided by the simultaneous truth and…
University College London（伦敦大学学院）；Verdure Imaging, Inc.（Verdure成像公司）；Queen’s University（女王大学）
Free Point Transformer (FPT) has been proposed as a data-driven, non-rigid point set registration approach using deep neural networks. As FPT does not assume constraints based on point vicinity or correspondence, it may be trained simply and in a flexible manner by minimizing an unsupervised loss based on the Chamfer Distance. This makes FPT amenable to real-world medical imaging applications where ground-truth deformations may be infeasible to obtain, or in scenarios where only a varying degree of completeness in the point sets to be aligned is available. To test the limit of the corresponden…
Università degli Studi di Milano - Bicocca（米兰比可卡大学）；Albert Ludwigs Universität Freiburg（阿尔伯特·路德维希·弗莱堡大学）；Universidad de Alcala de Henares（阿尔卡拉大学）
Point clouds registration is a fundamental step of many point clouds processing pipelines; however, most algorithms are tested on data that are collected ad-hoc and not shared with the research community. These data often cover only a very limited set of use cases; therefore, the results cannot be generalised. Public datasets proposed until now, taken individually, cover only a few kinds of environment and mostly a single sensor. For these reasons, we developed a benchmark, for localization and mapping applications, using multiple publicly available datasets. In this way, we are able to cover …
CAD-to-CT Registration of Cylindrical Objects via Ellipse-Based Axis Estimation
Department of Complex Systems, National Centre for Nuclear Research（复杂系统系，国家核研究中心）；ImagineRT sp. z o.o.（ImagineRT公司）；National Centre for Nuclear Research（国家核研究中心）
New York University（纽约大学）；NYU Abu Dhabi（纽约大学阿布扎比分校）；NYU Tandon School of Engineering（纽约大学坦登工程学院）
Point cloud registration is the process of aligning a pair of point sets via searching for a geometric transformation. Recent works leverage the power of deep learning for registering a pair of point sets. However, unfortunately, deep learning models often require a large number of ground truth labels for training. Moreover, for a pair of source and target point sets, existing deep learning mechanisms require explicitly designed encoders to extract both deep spatial features from unstructured point clouds and their spatial correlation representation, which is further fed to a decoder to regres…
Volumetric (3d) images are acquired for many scientific and biomedical purposes using imaging methods such as serial section microscopy, CT scans, and MRI. A frequent step in the analysis and reconstruction of such data is the alignment and registration of images that were acquired in succession along a spatial or temporal dimension. For example, in serial section electron microscopy, individual 2d sections are imaged via electron microscopy and then must be aligned to one another in order to produce a coherent 3d volume. State of the art approaches find image correspondences derived from patc…
Unsupervised Segmentation of Fetal Brain MRI using Deep Learning Cascaded Registration
Universitat Pompeu Fabra（庞培法布拉大学）；BCNatal；Institución Catalana de Investigación y Estudios Avanzados (ICREA)（加泰罗尼亚高级研究与发展研究院）；Centre for Biomedical Research on Rare Diseases (CIBERER)（罕见疾病生物医学研究中心）；European Commission, Joint Research Centre (JRC)（欧盟委员会联合研究中心）
Medical University of Graz（格拉茨医科大学）；Computer Algorithms for Medicine Laboratory（医学计算机算法实验室）；Graz University of Technology（格拉茨工业大学）
Image-based algorithmic software segmentation is an increasingly important topic in many medical fields. Algorithmic segmentation is used for medical three-dimensional visualization, diagnosis or treatment support, especially in complex medical cases. However, accessible medical databases are limited, and valid medical ground truth databases for the evaluation of algorithms are rare and usually comprise only a few images. Inaccuracy or invalidity of medical ground truth data and image-based artefacts also limit the creation of such databases, which is especially relevant for CT data sets of th…
We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive …
The recent application of deep learning in various areas of medical image analysis has brought excellent performance gains. In particular, technologies based on deep learning in medical image registration can outperform traditional optimisation-based registration algorithms both in registration time and accuracy. However, the U-net based architectures used in most of the image registration frameworks downscale the data, which removes global information and affects the deformation. In this paper, we present a densely connected convolutional architecture for deformable image registration. Our pr…
University College London（伦敦大学学院）；University of Bath（巴斯大学）；Google（谷歌）
In this paper we present a dense ground truth dataset of nonrigidly deforming real-world scenes. Our dataset contains both long and short video sequences, and enables the quantitatively evaluation for RGB based tracking and registration methods. To construct ground truth for the RGB sequences, we simultaneously capture Near-Infrared (NIR) image sequences where dense markers - visible only in NIR - represent ground truth positions. This allows for comparison with automatically tracked RGB positions and the formation of error metrics. Most previous datasets containing nonrigidly deforming sequen…
Washington University in St. Louis（圣路易斯华盛顿大学）；Mallinckrodt Institute of Radiology（马林克罗特放射学研究所）
Deep neural networks for medical image reconstruction are traditionally trained using high-quality ground-truth images as training targets. Recent work on Noise2Noise (N2N) has shown the potential of using multiple noisy measurements of the same object as an alternative to having a ground-truth. However, existing N2N-based methods are not suitable for learning from the measurements of an object undergoing nonrigid deformation. This paper addresses this issue by proposing the deformation-compensated learning (DeCoLearn) method for training deep reconstruction networks by compensating for object…
In this paper, we propose a transformer-based procedure for the efficient registration of non-rigid 3D point clouds. The proposed approach is data-driven and adopts for the first time the transformer architecture in the registration task. Our method is general and applies to different settings. Given a fixed template with some desired properties (e.g. skinning weights or other animation cues), we can register raw acquired data to it, thereby transferring all the template properties to the input geometry. Alternatively, given a pair of shapes, our method can register the first onto the second (…
University College London（伦敦大学学院）；InstaDeep；University College London Hospital NHS Foundation Trust（伦敦大学学院医院NHS基金会信托）
Organ morphology is a key indicator for prostate disease diagnosis and prognosis. For instance, In longitudinal study of prostate cancer patients under active surveillance, the volume, boundary smoothness and their changes are closely monitored on time-series MR image data. In this paper, we describe a new framework for forecasting prostate morphological changes, as the ability to detect such changes earlier than what is currently possible may enable timely treatment or avoiding unnecessary confirmatory biopsies. In this work, an efficient feature-based MR image registration is first developed…
Nonrigid Reconstruction of Freehand Ultrasound without a Tracker
Centre for Medical Image Computing（医学影像计算中心）；Wellcome/EPSRC Centre for Interventional and Surgical Sciences（惠康/工程与物理科学研究理事会介入与外科科学中心）；Department of Medical Physics and Biomedical Engineering（医学物理与生物医学工程系）；University College London（伦敦大学学院）；School of Biomedical Engineering & Imaging Sciences（生物医学工程与影像科学学院）；King’s College London（伦敦国王学院）
