NerT-CA: Efficient Dynamic Reconstruction from Sparse-view X-ray Coronary Angiography
Department of Mathematics and Computer Science, Eindhoven University of Technology, Eindhoven, The Netherlands（数学与计算机科学系，埃因霍温理工大学，埃因霍温，荷兰）；Philips Healthcare, Best, The Netherlands（飞利浦医疗，贝斯特，荷兰）；Department of Electrical Engineering, Eindhoven University of Technology, The Netherlands（电子工程系，埃因霍温理工大学，荷兰）
Kirsten W. H. Maas, Danny Ruijters, Nicola Pezzotti, Anna Vilanova
从 X 射线冠状动脉造影（CA）中重建冠状动脉的三维（3D）以及动态三维加时间（4D）结果，具有改善临床流程的潜力。然而，仍有多个挑战需要解决，最显著的是血管结构稀疏、背景与血管区分度差、视角稀疏以及扫描内运动。最先进的重建方法依赖耗时的手动分割或容易出错的自动分割，限制了临床可用性。近年来，基于神经辐射场（Neural Radiance Fields，NeRF）的方法在稀疏视角条件下的自动重建中展现出前景。但由于依赖基于 MLP 的表示，这些方法训练时间较长。我们提出 NerT-CA，这是一种结合神经表示与张量表示的混合方法，用于利用稀疏视角 CA 加速 4D 重建。在以往基于 NeRF 的工作基础上，我们将 CA 场景建模为低秩成分与稀疏成分的分解，使用快速张量场进行低秩静态重建，并使用神经场进行动态稀疏重建。我们的方法在训练时间和重建准确性方面均优于以往工作，能够从少至三个造影视角生成合理的重建结果。我们在具有代表性的 4D 体模数据集上对该方法进行了定量和定性验证。
英文摘要：
Three-dimensional (3D) and dynamic 3D+time (4D) reconstruction of coronary arteries from X-ray coronary angiography (CA) has the potential to improve clinical procedures. However, there are multiple challenges to be addressed, most notably, blood-vessel structure sparsity, poor background and blood vessel distinction, sparse-views, and intra-scan motion. State-of-the-art reconstruction approaches rely on time-consuming manual or error-prone automatic segmentations, limiting clinical usability. Recently, approaches based on Neural Radiance Fields (NeRF) have shown promise for automatic reconstructions in the sparse-view setting. However, they suffer from long training times due to their dependence on MLP-based representations. We propose NerT-CA, a hybrid approach of Neural and Tensorial representations for accelerated 4D reconstructions with sparse-view CA. Building on top of the previous NeRF-based work, we model the CA scene as a decomposition of low-rank and sparse components, utilizing fast tensorial fields for low-rank static reconstruction and neural fields for dynamic sparse reconstruction. Our approach outperforms previous works in both training time and reconstruction accuracy, yielding reasonable reconstructions from as few as three angiogram views. We validate our approach quantitatively and qualitatively on representative 4D phantom datasets.
Queen Mary University of London（伦敦玛丽女王大学）；Barts Health NHS Trust（巴特国民保健信托基金会）
X-ray coronary angiography (XCA) is used to assess coronary artery disease and provides valuable information on lesion morphology and severity. However, XCA images are 2D and therefore limit visualisation of the vessel. 3D reconstruction of coronary vessels is possible using multiple views, however lumen border detection in current software is performed manually resulting in limited reproducibility and slow processing time. In this study we propose 3DAngioNet, a novel deep learning (DL) system that enables rapid 3D vessel mesh reconstruction using 2D XCA images from two views. Our approach lea…
NeRF-CA: Dynamic Reconstruction of X-ray Coronary Angiography with Extremely Sparse-views
Department of Mathematics & Computer Science, Eindhoven University of Technology（数学与计算机科学系，埃因霍温理工大学）；Philips Healthcare（飞利普医疗）；Department of Electrical Engineering, Eindhoven University of Technology（电子工程系，埃因霍温理工大学）；ASML（ASML公司）
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；University Medical Center Utrecht（乌得勒支大学医学中心）；Amsterdam University Medical Center（阿姆斯特丹大学医学中心）；Utrecht University（乌得勒支大学）
In patients with obstructive coronary artery disease, the functional significance of a coronary artery stenosis needs to be determined to guide treatment. This is typically established through fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA). We present a method for automatic and non-invasive detection of patients requiring ICA, employing deep unsupervised analysis of complete coronary arteries in cardiac CT angiography (CCTA) images. We retrospectively collected CCTA scans of 187 patients, 137 of them underwent invasive FFR measurement in 192 dif…
Tsinghua University（清华大学）；Anzhen Hospital, Capital Medical University（首都医科大学附属安贞医院）；Capital Medical University（首都医科大学）；The First Affiliated Hospital of Zhengzhou University（郑州大学第一附属医院）
Cardiac coronary angiography is a major technology to assist doctors during cardiac interventional surgeries. Under the exposure of X-ray radiation, doctors inject contrast agents through catheters to determine the position and status of coronary vessels in real time. To get a coronary angiography video with a high frame rate, the doctor needs to increase the exposure frequency and intensity of the X-ray. This will inevitably increase the X-ray harm to both patients and surgeons. In this work, we innovatively utilize a deep-learning based video interpolation algorithm to interpolate coronary a…
Image Sciences Institute（影像科学研究所）；University Medical Center Utrecht（乌得勒支大学医学中心）；Department of Radiology（放射科）
Detection of coronary artery stenosis in coronary CT angiography (CCTA) requires highly personalized surface meshes enclosing the coronary lumen. In this work, we propose to use graph convolutional networks (GCNs) to predict the spatial location of vertices in a tubular surface mesh that segments the coronary artery lumen. Predictions for individual vertex locations are based on local image features as well as on features of neighboring vertices in the mesh graph. The method was trained and evaluated using the publicly available Coronary Artery Stenoses Detection and Quantification Evaluation …
Fudan University（复旦大学）；Shanghai East Hospital, Tongji University（同济大学附属东方医院）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）
We proposed a novel unsupervised methodology named Disarranged Zone Learning (DZL) to automatically recognize stenosis in coronary angiography. The methodology firstly disarranges the frames in a video, secondly it generates an effective zone and lastly trains an encoder-decoder GRU model to learn the capability to recover disarranged frames. The breakthrough of our study is to discover and validate the Sequence Intensity (Recover Difficulty) is a measure of Coronary Artery Stenosis Status. Hence, the prediction accuracy of DZL is used as an approximator of coronary stenosis indicator. DZL is …
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；University Medical Center Utrecht（乌得勒支大学医学中心）；Utrecht University（乌得勒支大学）；Amsterdam University Medical Center（阿姆斯特丹大学医学中心）
Treatment of patients with obstructive coronary artery disease is guided by the functional significance of a coronary artery stenosis. Fractional flow reserve (FFR), measured during invasive coronary angiography (ICA), is considered the gold standard to define the functional significance of a coronary stenosis. Here, we present a method for non-invasive detection of patients with functionally significant coronary artery stenosis, combining analysis of the coronary artery tree and the left ventricular (LV) myocardium in cardiac CT angiography (CCTA) images. We retrospectively collected CCTA sca…
Coronary Heart Disease (CHD) is a leading cause of death in the modern world. The development of modern analytical tools for diagnostics and treatment of CHD is receiving substantial attention from the scientific community. Deep learning-based algorithms, such as segmentation networks and detectors, play an important role in assisting medical professionals by providing timely analysis of a patient's angiograms. This paper focuses on X-Ray Coronary Angiography (XCA), which is considered to be a "gold standard" in the diagnosis and treatment of CHD. First, we describe publicly available datasets…
CADICA: a new dataset for coronary artery disease detection by using invasive coronary angiography
University of Málaga（马拉加大学）；Instituto de Investigación Biomédica de Málaga y Plataforma en Nanomedicina-IBIMA Plataforma BIONAND（马拉加生物医学与纳米医学平台研究所-IBIMA BIONAND平台）；Centro de Investigación Biomédica en Red de Enfermedades Cardiovasculares (CIBERCV)（心血管疾病网络生物医学研究中心）；Instituto de Salud Carlos III (ISCIII)（卡洛斯三世卫生研究所）
Tsinghua University（清华大学）；Beijing Anzhen Hospital, Capital Medical University（首都医科大学附属北京安贞医院）；The First Affiliated Hospital of Zhengzhou University（郑州大学第一附属医院）
Coronary angiography is an indispensable assistive technique for cardiac interventional surgery. Segmentation and extraction of blood vessels from coronary angiography videos are very essential prerequisites for physicians to locate, assess and diagnose the plaques and stenosis in blood vessels. This article proposes a new video segmentation framework that can extract the clearest and most comprehensive coronary angiography images from a video sequence, thereby helping physicians to better observe the condition of blood vessels. This framework combines a 3D convolutional layer to extract spati…
DeepCA: Deep Learning-based 3D Coronary Artery Tree Reconstruction from Two 2D Non-simultaneous X-ray Angiography Projections
University of Oxford（牛津大学）；Institute of Biomedical Engineering（生物医学工程研究所）；Division of Cardiovascular Medicine（心血管医学部）；Radcliffe Department of Medicine（拉德克利夫医学系）；Oxford Acute Vascular Imaging Centre（牛津急性血管成像中心）
Bioinformatics Institute, Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局生物信息学研究所）；Hwa Chong Institution (College)（华侨中学（高中部））；National Heart Centre Singapore（新加坡国家心脏中心）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；School of Computing, National University of Singapore（新加坡国立大学计算学院）；Singapore Eye Research Institute (SERI)（新加坡眼科研究所）；Rehabilitation Research Institute of Singapore（新加坡康复研究所）；Institute of Bioengineering and Nanotechnology, A*STAR（新加坡科技研究局生物工程与纳米技术研究所）
The evaluation of obstructions (stenosis) in coronary arteries is currently done by a physician's visual assessment of coronary angiography video sequences. It is laborious, and can be susceptible to interobserver variation. Prior studies have attempted to automate this process, but few have demonstrated an integrated suite of algorithms for the end-to-end analysis of angiograms. We report an automated analysis pipeline based on deep learning to rapidly and objectively assess coronary angiograms, highlight coronary vessels of interest, and quantify potential stenosis. We propose a 3-stage auto…
Voxel2Hemodynamics: An End-to-end Deep Learning Method for Predicting Coronary Artery Hemodynamics
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；Centre for Perceptual and Interactive Intelligence, the Chinese University of Hong Kong（香港中文大学感知与交互智能研究中心）；SenseTime Research（商汤科技研究院）；the Chinese University of Hong Kong（香港中文大学）
Vessel stenosis is a major risk factor in cardiovascular diseases (CVD). To analyze the degree of vessel stenosis for supporting the treatment management, extraction of coronary artery area from Computed Tomographic Angiography (CTA) is regarded as a key procedure. However, manual segmentation by cardiologists may be a time-consuming task, and present a significant inter-observer variation. Although various computer-aided approaches have been developed to support segmentation of coronary arteries in CTA, the results remain unreliable due to complex attenuation appearance of plaques, which are …
Instituto Superior Técnico（高等技术学院）；University of Lisbon（里斯本大学）；CCUL（里斯本大学中心学院）；Lisbon School of Medicine（里斯本医学院）
Coronary artery disease leading up to stenosis, the partial or total blocking of coronary arteries, is a severe condition that affects millions of patients each year. Automated identification and classification of stenosis severity from minimally invasive procedures would be of great clinical value, but existing methods do not match the accuracy of experienced cardiologists, due to the complexity of the task. Although a number of computational approaches for quantitative assessment of stenosis have been proposed to date, the performance of these methods is still far from the required levels fo…
Zhengzhou University of Light Industry（郑州轻工业大学）；University of Southern Mississippi（南密西西比大学）；Michigan Technological University（密歇根理工大学）；New York Presbyterian Queens Hospital（纽约长老会皇后医院）；The First Affiliated Hospital of Nanjing Medical University（南京医科大学第一附属医院）
Automatic identification of proper image frames at the end-diastolic (ED) and end-systolic (ES) frames during the review of invasive coronary angiograms (ICA) is important to assess blood flow during a cardiac cycle, reconstruct the 3D arterial anatomy from bi-planar views, and generate the complementary fusion map with myocardial images. The current identification method primarily relies on visual interpretation, making it not only time-consuming but also less reproducible. In this paper, we propose a new method to automatically identify angiographic image frames associated with the ED and ES…
The Ohio State University College of Medicine（俄亥俄州立大学医学院）
We propose a fully automated algorithm based on a deep learning framework enabling screening of a coronary computed tomography angiography (CCTA) examination for confident detection of the presence or absence of coronary artery atherosclerosis. The system starts with extracting the coronary arteries and their branches from CCTA datasets and representing them with multi-planar reformatted volumes; pre-processing and augmentation techniques are then applied to increase the robustness and generalization ability of the system. A 3-dimensional convolutional neural network (3D-CNN) is utilized to mo…
Korea Advanced Institute of Science and Technology（韩国科学技术院）；University of Ulsan College of Medicine（蔚山大学医学院）；Asan Medical Center（峨山医学中心）
In this paper, we proposed T-Net containing a small encoder-decoder inside the encoder-decoder structure (EDiED). T-Net overcomes the limitation that U-Net can only have a single set of the concatenate layer between encoder and decoder block. To be more precise, the U-Net symmetrically forms the concatenate layers, so the low-level feature of the encoder is connected to the latter part of the decoder, and the high-level feature is connected to the beginning of the decoder. T-Net arranges the pooling and up-sampling appropriately during the encoder process, and likewise during the decoding proc…
Peking University（北京大学）；Peking University First Hospital（北京大学第一医院）；MGH/BWH Center for Clinical Data Science（麻省总医院/布莱根妇女医院临床数据科学中心）
Segmenting coronary arteries is challenging, as classic unsupervised methods fail to produce satisfactory results and modern supervised learning (deep learning) requires manual annotation which is often time-consuming and can some time be infeasible. To solve this problem, we propose a knowledge transfer based shape-consistent generative adversarial network (SC-GAN), which is an annotation-free approach that uses the knowledge from publicly available annotated fundus dataset to segment coronary arteries. The proposed network is trained in an end-to-end fashion, generating and segmenting synthe…
CAD-RADS scoring of coronary CT angiography with Multi-Axis Vision Transformer: a clinically-inspired deep learning pipeline
University of Pavia（帕维亚大学）；Centro Cardiologico Monzino IRCCS（蒙齐诺心脏病学中心IRCCS）；University of Milan（米兰大学）；IRCCS Istituti Clinici Scientifici Maugeri（IRCCS莫杰里科学临床研究所）
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；School of Continuing Education, Shanghai Jiao Tong University（上海交通大学继续教育学院）；Shanghai Jiao Tong University Affiliated Sixth People’s Hospital（上海交通大学附属第六人民医院）；Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University（上海交通大学医学院附属仁济医院）；Center for Advanced Neuroimaging, University of California, Riverside（加州大学河滨分校高级神经影像中心）
Although robust PCA has been increasingly adopted to extract vessels from X-ray coronary angiography (XCA) images, challenging problems such as inefficient vessel-sparsity modelling, noisy and dynamic background artefacts, and high computational cost still remain unsolved. Therefore, we propose a novel robust PCA unrolling network with sparse feature selection for super-resolution XCA vessel imaging. Being embedded within a patch-wise spatiotemporal super-resolution framework that is built upon a pooling layer and a convolutional long short-term memory network, the proposed network can not onl…
CardioSyntax: end-to-end SYNTAX score prediction -- dataset, benchmark and method
Sber AI Lab（Sber AI实验室）；Tyumen Cardiology Research Center（泰姆普斯心脏病学研究中心）；Tomsk National Research Medical Center of Russian Academy of Science（托木斯克俄罗斯科学院国家医学研究中心）
ImageCAS: A Large-Scale Dataset and Benchmark for Coronary Artery Segmentation based on Computed Tomography Angiography Images
Guangdong University of Technology（广东工业大学）；Guangdong Provincial People’s Hospital (Guangdong Academy of Medical Sciences)（广东省人民医院（广东省医学科学院））；Southern Medical University（南方医科大学）；Missouri University of Science and Technology（密苏里科技大学）
The purpose of this study is to present a new semi-automated methodology for three-dimensional (3D) reconstruction of coronary arteries and their plaque morphology using Computed Tomography Angiography (CTA) images. The methodology is summarized in seven stages: pre-processing of the acquired CTA images, extraction of the vessel tree centerline, estimation of a weight function for lumen, outer wall and calcified plaque, lumen segmentation, outer wall segmentation, plaque detection, and finally 3D surfaces construction. The methodology was evaluated using both expert manual annotations and esti…
Multivessel Coronary Artery Segmentation and Stenosis Localisation using Ensemble Learning
Birmingham City University（伯明翰城市大学）；University of the West of England (UWE)（西英格兰大学）；Institute of Accounting and Administration, University of Aveiro（阿威罗大学会计与行政学院）；NUS High School of Math and Science（新加坡国立大学数学与科学高中）；Qatar University（卡塔尔大学）；Information Technology University（信息技术大学）；Bristol Heart Institute, University of Bristol（布里斯托大学布里斯托心脏研究所）
Purpose: To study the accuracy of motion information extracted from beat-to-beat 3D image-based navigators (3D iNAVs) collected using a variable-density cones trajectory with different combinations of spatial resolutions and scan acceleration factors. Methods: Fully sampled, breath-held 4.4 mm 3D iNAV datasets for six respiratory phases are acquired in a volunteer. Ground truth translational and nonrigid motion information is derived from these datasets. Subsequently, the motion estimates from synthesized undersampled 3D iNAVs with isotropic spatial resolutions of 4.4 mm (acceleration factor =…
X-ray coronary angiography (XCA) is a principal approach employed for identifying coronary disorders. Deep learning-based networks have recently shown tremendous promise in the diagnosis of coronary disorder from XCA scans. A deep learning-based edge adaptive instance normalization style transfer technique for segmenting the coronary arteries, is presented in this paper. The proposed technique combines adaptive instance normalization style transfer with the dense extreme inception network and convolution block attention module to get the best artery segmentation performance. We tested the prop…
Physics-Informed Hemodynamic Modeling for Data-Free Prediction and Sparse-Data Assimilation
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）；West China Hospital, Sichuan University（四川大学华西医院）
Background: RSN whole-heart CMRA is a technique that estimates and corrects for respiratory motion. However, RSN has been limited to a 1D rigid correction which is often insufficient for patients with complex respiratory patterns. The goal of this work is therefore to improve the robustness and quality of 3D radial CMRA by incorporating both 3D motion information and nonrigid intra-acquisition correction of the data into a framework called focused navigation (fNAV). Methods: We applied fNAV to 500 data sets from a numerical simulation, 22 healthy volunteers, and 549 cardiac patients. We compar…
Purpose: To rapidly reconstruct undersampled 3D non-Cartesian image-based navigators (iNAVs) using an unrolled deep learning (DL) model for non-rigid motion correction in coronary magnetic resonance angiography (CMRA). Methods: An unrolled network is trained to reconstruct beat-to-beat 3D iNAVs acquired as part of a CMRA sequence. The unrolled model incorporates a non-uniform FFT operator to perform the data consistency operation, and the regularization term is learned by a convolutional neural network (CNN) based on the proximal gradient descent algorithm. The training set includes 6,000 3D i…
Coronary heart disease (CHD) is the leading cause of adult death in the United States and worldwide, and for which the coronary angiography procedure is the primary gateway for diagnosis and clinical management decisions. The standard-of-care for interpretation of coronary angiograms depends upon ad-hoc visual assessment by the physician operator. However, ad-hoc visual interpretation of angiograms is poorly reproducible, highly variable and bias prone. Here we show for the first time that fully-automated angiogram interpretation to estimate coronary artery stenosis is possible using a sequenc…
Cross-Modal Contrastive Learning of ECG and Angiography Representations for Severe Stenosis Classification
Chair for AI in Healthcare and Medicine, Technical University of Munich and TUM University Hospital（人工智能在医疗与医学中的研究所，慕尼黑技术大学及慕尼黑大学医院）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML））；Department of Internal Medicine, TUM University Hospital（慕尼黑大学医院内科学系）
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Siemens Healthcare GmbH（西门子医疗有限公司）；University Clinic Frankfurt（法兰克福大学诊所）；Cardioangiological Centrum Bethanien（贝塔尼恩心血管中心）
Assessing coronary artery plaque segments in coronary CT angiography scans is an important task to improve patient management and clinical outcomes, as it can help to decide whether invasive investigation and treatment are necessary. In this work, we present three machine learning approaches capable of performing this task. The first approach is based on radiomics, where a plaque segmentation is used to calculate various shape-, intensity- and texture-based features under different image transformations. A second approach is based on deep learning and relies on centerline extraction as sole pr…
Philips Healthcare（飞利浦医疗）；Philips Research（飞利浦研究院）；Semmelweis University（塞梅尔魏斯大学）；Queen’s University（女王大学）；South Eastern Health and Social Care Trust（东南健康与社会护理信托基金）
We introduce a functional for the learning of an optimal database for patch-based image segmentation with application to coronary lumen segmentation from coronary computed tomography angiography (CCTA) data. The proposed functional consists of fidelity, sparseness and robustness to small-variations terms and their associated weights. Existing work address database optimization by prototype selection aiming to optimize the database by either adding or removing prototypes according to a set of predefined rules. In contrast, we formulate the database optimization task as an energy minimization pr…
