MRAnnotator: multi-Anatomy and many-Sequence MRI segmentation of 44 structures
BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院生物医学工程与影像研究所）；Department of Diagnostic, Molecular, and Interventional Radiology, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院诊断、分子与介入放射学系）；Rotterdam School of Management, Erasmus University Rotterdam（伊拉斯姆斯大学鹿特丹管理学院）；East River Medical Imaging（东河医学影像中心）；Department of Radiology, Cornell Medicine（康奈尔医学院放射学系）；Windreich Department of Artificial Intelligence and Human Health, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院温赖克人工智能与人类健康系）
Alexander Zhou, Zelong Liu, Andrew Tieu, Nikhil Patel, Sean Sun, Anthony Yang, Peter Choi, Hao-Chih Lee, Mickael Tordjman, Louisa Deyer, Yunhao Mei, Valentin Fa… 展开作者
Alexander Zhou, Zelong Liu, Andrew Tieu, Nikhil Patel, Sean Sun, Anthony Yang, Peter Choi, Hao-Chih Lee, Mickael Tordjman, Louisa Deyer, Yunhao Mei, Valentin Fauveau, George Soultanidis, Bachir Taouli, Mingqian Huang, Amish Doshi, Zahi A. Fayad, Timothy Deyer, Xueyan Mei
发表信息：
Radiology Advances, Volume 2, Issue 1, January 2025
In this retrospective study, we annotated 44 structures on two datasets: an internal dataset of 1,518 MRI sequences from 843 patients at the Mount Sinai Health System, and an external dataset of 397 MRI sequences from 263 patients for benchmarking. The internal dataset trained the nnU-Net model MRAnnotator, which demonstrated strong generalizability on the external dataset. MRAnnotator outperformed existing models such as TotalSegmentator MRI and MRSegmentator on both datasets, achieving an overall average Dice score of 0.878 on the internal dataset and 0.875 on the external set. Model weights are available on GitHub, and the external test set can be shared upon request.
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Tencent AI Lab（腾讯人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
Recently, deep neural networks have greatly advanced undersampled Magnetic Resonance Image (MRI) reconstruction, wherein most studies follow the one-anatomy-one-network fashion, i.e., each expert network is trained and evaluated for a specific anatomy. Apart from inefficiency in training multiple independent models, such convention ignores the shared de-aliasing knowledge across various anatomies which can benefit each other. To explore the shared knowledge, one naive way is to combine all the data from various anatomies to train an all-round network. Unfortunately, despite the existence of th…
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
University of Pittsburgh（匹兹堡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cedars-Sinai Medical Center（西德萨斯医疗中心）；Mayo Clinic Arizona（梅奥诊所亚利桑那分部）
McGill University（麦吉尔大学）；MILA (Québec Artificial Intelligence Institute)（MILA（魁北克人工智能研究所））；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
We propose a hierarchically structured variational inference model for accurately disentangling observable evidence of disease (e.g. brain lesions or atrophy) from subject-specific anatomy in brain MRIs. With flexible, partially autoregressive priors, our model (1) addresses the subtle and fine-grained dependencies that typically exist between anatomical and pathological generating factors of an MRI to ensure the clinical validity of generated samples; (2) preserves and disentangles finer pathological details pertaining to a patient's disease state. Additionally, we experiment with an alternat…
MIT（麻省理工学院）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Fetal MRI is heavily constrained by unpredictable and substantial fetal motion that causes image artifacts and limits the set of viable diagnostic image contrasts. Current mitigation of motion artifacts is predominantly performed by fast, single-shot MRI and retrospective motion correction. Estimation of fetal pose in real time during MRI stands to benefit prospective methods to detect and mitigate fetal motion artifacts where inferred fetal motion is combined with online slice prescription with low-latency decision making. Current developments of deep reinforcement learning (DRL), offer a nov…
Contrastive Anatomy-Contrast Disentanglement: A Domain-General MRI Harmonization Method
Chair for AI for Image-Guided Diagnosis and Therapy, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany（人工智能辅助影像引导诊断与治疗研究所，慕尼黑技术大学（TUM）及慕尼黑技术大学医院，慕尼黑，德国）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心，慕尼黑，德国）；Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany（人工智能在医疗与健康领域研究所，慕尼黑技术大学（TUM）及慕尼黑技术大学医院，慕尼黑，德国）；Department of Radiation Oncology, TUM University Hospital, Munich, Germany（放射肿瘤科，慕尼黑技术大学医院，慕尼黑，德国）；Chair for Computer Vision and Artificial Intelligence, Technical University of Munich (TUM), Munich, Germany（计算机视觉与人工智能研究所，慕尼黑技术大学（TUM），慕尼黑，德国）；Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, CA, USA（斯坦福大学人工智能在医学与影像中心，斯坦福大学，斯坦福，CA，美国）；Deutsches Konsortium für Translationale Krebsforschung (DKTK), Partner Site Munich, Munich, Germany（德国转化癌症研究联盟（DKTK），慕尼黑合作伙伴站点，慕尼黑，德国）；Institute of Radiation Medicine (IRM), Department of Radiation Sciences (DRS), Helmholtz Center Munich, Munich, Germany（放射医学研究所（IRM），辐射科学部门（DRS），海德堡慕尼黑研究中心，慕尼黑，德国）
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for in…
The University of Queensland（昆士兰大学）；CSIRO（联邦科学与工业研究组织）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Deep neural networks have been extensively studied for undersampled MRI reconstruction. While achieving state-of-the-art performance, they are trained and deployed specifically for one anatomy with limited generalization ability to another anatomy. Rather than building multiple models, a universal model that reconstructs images across different anatomies is highly desirable for efficient deployment and better generalization. Simply mixing images from multiple anatomies for training a single network does not lead to an ideal universal model due to the statistical shift among datasets of various…
Anatomy-Informed Deep Learning and Radiomics for Automated Neurofibroma Segmentation in Whole-Body MRI
Institute for Applied Medical Informatics（应用医学信息学研究所）；Institute of Computational Neuroscience（计算神经科学研究所）；Center for Biomedical Artificial Intelligence (bAIome)（生物医学人工智能中心）；University Medical Center Hamburg-Eppendorf（汉堡埃彭多夫大学医学中心）；Department of Diagnostic and Interventional Radiology and Nuclear Medicine（诊断与介入放射学及核医学部门）；Department of Neurology（神经内科部门）
Full-Head Segmentation of MRI with Abnormal Brain Anatomy: Model and Data Release
The City College of New York（纽约市立学院）；New York University（纽约大学）；Georgetown University（乔治城大学）；University of North Carolina School of Medicine（北卡罗来纳大学医学院）；Soterix Medical
EPFL（洛桑联邦理工学院）；Bilkent University（比尔肯特大学）；National University of Singapore（新加坡国立大学）
In the area of magnetic resonance imaging (MRI), an extensive range of non-linear reconstruction algorithms have been proposed that can be used with general Fourier subsampling patterns. However, the design of these subsampling patterns has typically been considered in isolation from the reconstruction rule and the anatomy under consideration. In this paper, we propose a learning-based framework for optimizing MRI subsampling patterns for a specific reconstruction rule and anatomy, considering both the noiseless and noisy settings. Our learning algorithm has access to a representative set of t…
Anatomy-Guided Multitask Learning for MRI-Based Classification of Placenta Accreta Spectrum and its Subtypes
Sun Yat-sen University（中山大学）；The Third Affiliated Hospital of Guangzhou Medical University（广州医科大学附属第三医院）；Hainan General Hospital, Hainan Medical University（海南医科大学附属海南医院）
Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRI
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（马萨诸塞州总医院阿西诺拉·A·马丁诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（马萨诸塞州总医院）
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
Vision transformers, with their ability to more efficiently model long-range context, have demonstrated impressive accuracy gains in several computer vision and medical image analysis tasks including segmentation. However, such methods need large labeled datasets for training, which is hard to obtain for medical image analysis. Self-supervised learning (SSL) has demonstrated success in medical image segmentation using convolutional networks. In this work, we developed a \underline{s}elf-distillation learning with \underline{m}asked \underline{i}mage modeling method to perform SSL for vision \u…
Philips Research（飞利浦研究院）；Skolkovo Institute of Science and Technology（斯科尔科沃科技学院）
Domain Adaptation (DA) methods are widely used in medical image segmentation tasks to tackle the problem of differently distributed train (source) and test (target) data. We consider the supervised DA task with a limited number of annotated samples from the target domain. It corresponds to one of the most relevant clinical setups: building a sufficiently accurate model on the minimum possible amount of annotated data. Existing methods mostly fine-tune specific layers of the pretrained Convolutional Neural Network (CNN). However, there is no consensus on which layers are better to fine-tune, e.…
Technische Universität München（慕尼黑工业大学）；ScintHealth GmbH（ScintHealth有限公司）；Reply SpA（Reply股份公司）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）；Helmholtz AI（亥姆霍兹人工智能研究所）
CT and MRI are two of the most informative modalities in spinal diagnostics and treatment planning. CT is useful when analysing bony structures, while MRI gives information about the soft tissue. Thus, fusing the information of both modalities can be very beneficial. Registration is the first step for this fusion. While the soft tissues around the vertebra are deformable, each vertebral body is constrained to move rigidly. We propose a weakly-supervised deep learning framework that preserves the rigidity and the volume of each vertebra while maximizing the accuracy of the registration. To achi…
Facebook AI Research (FAIR)（Facebook人工智能研究院（FAIR））；NYU School of Medicine（纽约大学医学院）
The slow acquisition speed of magnetic resonance imaging (MRI) has led to the development of two complementary methods: acquiring multiple views of the anatomy simultaneously (parallel imaging) and acquiring fewer samples than necessary for traditional signal processing methods (compressed sensing). While the combination of these methods has the potential to allow much faster scan times, reconstruction from such undersampled multi-coil data has remained an open problem. In this paper, we present a new approach to this problem that extends previously proposed variational methods by learning ful…
Automated surgical planning with nnU-Net: delineation of the anatomy in hepatobiliary phase MRI
The Netherlands Cancer Institute – Antoni van Leeuwenhoek（荷兰癌症研究所 – 安东尼·范·列文虎克）；National Center for Tumor Diseases (NCT), NCT/UCC Dresden（国家肿瘤疾病中心（NCT），NCT/UCC 德累斯顿）；DKFZ（德国癌症研究中心）；Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology（德累斯顿工业大学医学院及卡尔·古斯塔夫·卡鲁斯大学医院）；The Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden University of Technology（德累斯顿工业大学触觉互联网与人机协同中心）；University of Twente（特文特大学）
Unsupervised Anomaly Detection in Brain MRI via Disentangled Anatomy Learning
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China.（自动化与智能感知学院，上海交通大学，中国）；School of Computer Science, The University of Sydney, Sydney, Australia.（计算机科学学院，悉尼大学，澳大利亚）；Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.（放射科，仁济医院，医学院，上海交通大学，中国）；Biomedical Engineering Department, Tulane University, New Orleans, USA.（生物医学工程系， Tulane 大学，美国新奥尔良）
University College London（伦敦大学学院）；Centre for Medical Image Computing（医学图像计算中心）
Accurate segmentation of the right ventricle (RV) is a crucial step in the assessment of the ventricular structure and function. Yet, due to its complex anatomy and motion segmentation of the RV has not been as largely studied as the left ventricle. This paper presents a fully automatic method for the segmentation of the RV in cardiac magnetic resonance images (MRI). The method uses a coarse-to-fine segmentation strategy in combination with a multi-atlas propagation segmentation framework. Based on a cross correlation metric, our method selects the best atlases for propagation allowing the ref…
Rethinking Convergence in Deep Learning: The Predictive-Corrective Paradigm for Anatomy-Informed Brain MRI Segmentation
School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院）；Key Laboratory of Signal Detection and Processing, Xinjiang University（新疆大学信号检测与处理重点实验室）；Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center（新疆多模态智能处理与信息安全工程技术研究中心）；National Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）
A generalisable head MRI defacing pipeline: Evaluation on 2,566 meningioma scans
School of Biomedical Engineering & Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与成像科学学校）；Department of Neurosurgery, King’s College Hospital NHS Foundation Trust（伦敦国王学院医院神经外科部门）
Royal Brompton Hospital（皇家布朗普顿医院）；National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Fudan University（复旦大学）；Shanghai Jiao Tong University（上海交通大学）；University of Lincoln（林肯大学）；City University London（伦敦城市大学）
Purpose: Atrial fibrillation (AF) is the most common cardiac arrhythmia and is correlated with increased morbidity and mortality. It is associated with atrial fibrosis, which may be assessed non-invasively using late gadolinium-enhanced (LGE) magnetic resonance imaging (MRI) where scar tissue is visualised as a region of signal enhancement. In this study, we proposed a novel fully automatic pipeline to achieve an accurate and objective atrial scarring segmentation and assessment of LGE MRI scans for the AF patients. Methods: Our fully automatic pipeline uniquely combined: (1) a multi-atlas bas…
Brightness-Invariant Tracking Estimation in Tagged MRI
Johns Hopkins University（约翰霍普金斯大学）；University of Maryland School of Medicine（马里兰大学医学院）；Uniformed Services University of the Health Sciences（统一服务卫生科学大学）；The Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc.（亨利·M·杰克逊军事医学促进基金会）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Washington University（华盛顿大学）；University of South Carolina（南卡罗来纳大学）
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（汉堡工业大学医疗技术与智能系统研究所）；jung diagnostics GmbH（jung diagnostics GmbH公司）；Hamburg University of Technology（汉堡工业大学）
Lesion detection in brain Magnetic Resonance Images (MRIs) remains a challenging task. MRIs are typically read and interpreted by domain experts, which is a tedious and time-consuming process. Recently, unsupervised anomaly detection (UAD) in brain MRI with deep learning has shown promising results to provide a quick, initial assessment. So far, these methods only rely on the visual appearance of healthy brain anatomy for anomaly detection. Another biomarker for abnormal brain development is the deviation between the brain age and the chronological age, which is unexplored in combination with …
Medical Resonance Imaging or MRI is a medical image processing technique that used radio waves to scan the body. It is a tomographic imaging technique, principally used in the field of radiology. With the advantage of being a painless diagnostic procedure, MRI allows medical personnel to illustrate clear pictures of the anatomy and the physiological processes occurring in the body, thus allowing early detection and treatment of diseases. These images, combined with image processing techniques may be used in the detection of tumors, difficult to identify with the naked eye. This digital assignm…
The human cortical layer exhibits a convoluted morphology that is unique to each individual. Conventional volumetric fMRI processing schemes take for granted the rich information provided by the underlying anatomy. We present a method to study fMRI data on subject-specific cerebral hemisphere cortex (CHC) graphs, which encode the cortical morphology at the resolution of voxels in 3-D. We study graph spectral energy metrics associated to fMRI data of 100 subjects from the Human Connectome Project database, across seven tasks. Experimental results signify the strength of CHC graphs' Laplacian ei…
University of Florida（佛罗里达大学）；Georgia State University（佐治亚州立大学）
Generating multi-contrasts/modal MRI of the same anatomy enriches diagnostic information but is limited in practice due to excessive data acquisition time. In this paper, we propose a novel deep-learning model for joint reconstruction and synthesis of multi-modal MRI using incomplete k-space data of several source modalities as inputs. The output of our model includes reconstructed images of the source modalities and high-quality image synthesized in the target modality. Our proposed model is formulated as a variational problem that leverages several learnable modality-specific feature extract…
Probabilistic modelling has been an essential tool in medical image analysis, especially for analyzing brain Magnetic Resonance Images (MRI). Recent deep learning techniques for estimating high-dimensional distributions, in particular Variational Autoencoders (VAEs), opened up new avenues for probabilistic modeling. Modelling of volumetric data has remained a challenge, however, because constraints on available computation and training data make it difficult effectively leverage VAEs, which are well-developed for 2D images. We propose a method to model 3D MR brain volumes distribution by combi…
Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina at Chapel Hill（放射科与生物医学研究成像中心（BRIC）、北卡罗来纳大学教堂山分校）；Lampe Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University（Lampe联合生物医学工程部门、北卡罗来纳大学教堂山分校和北卡罗来纳州立大学）
University of Central Florida（中佛罗里达大学）；Kyungpook National University（庆北国立大学）
Medical image segmentation being a substantial component of image processing plays a significant role to analyze gross anatomy, to locate an infirmity and to plan the surgical procedures. Segmentation of brain Magnetic Resonance Imaging (MRI) is of considerable importance for the accurate diagnosis. However, precise and accurate segmentation of brain MRI is a challenging task. Here, we present an efficient framework for segmentation of brain MR images. For this purpose, Gabor transform method is used to compute features of brain MRI. Then, these features are classified by using four different …
Evaluation of 3D Counterfactual Brain MRI Generation
Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA Department of Radiology, Stanford University, Stanford, CA, USA Dept. of Psychiatry \& Behavioral Sciences, Stanford University, Stanford, CA, USA Department of Computer Science, Stanford University, Stanford, CA, USA Department of Bioengineering, Stanford University, Stanford, CA, USA Department of Electrical Engineering, Stanford University, Stanford, CA, USA
Learning accurate rigid registration for longitudinal brain MRI from synthetic data
1 Division of Biomedical Imaging, KTH Royal Institute of Technology, Huddinge, Sweden 2 Athinoula A.\ Martinos Center for Biomedical Imaging, Charlestown, USA 3 Department of Radiology, Massachusetts General Hospital, Boston, USA 4 Department of Radiology, Harvard Medical School, Boston, USA 5 Computer Science \& Artificial Intelligence Laboratory, MIT, Cambridge, USA
University of Calcutta（加尔各答大学）；West Bengal University of Technology（西孟加拉技术大学）
Tumor segmentation from magnetic resonance imaging (MRI) data is an important but time consuming manual task performed by medical experts. Automating this process is a challenging task because of the high diversity in the appearance of tumor tissues among different patients and in many cases similarity with the normal tissues. MRI is an advanced medical imaging technique providing rich information about the human soft-tissue anatomy. There are different brain tumor detection and segmentation methods to detect and segment a brain tumor from MRI images. These detection and segmentation approache…
Counterfactual MRI Data Augmentation using Conditional Denoising Diffusion Generative Models
Instituto Superior Técnico（高等技术学院）；Universidade de Lisboa（里斯本大学）；Digital Surgery LAB（数字外科实验室）；Champalimaud Foundation（尚帕利莫基金会）；Faculdade de Medicina, Universidade de Lisboa（里斯本大学医学院）；Instituto de Telecomunicações（电信研究所）
University College London（伦敦大学学院）；Technion -- Israel Institute of Technology（以色列理工学院）；Heriot-Watt University（赫瑞-瓦特大学）
Medical imaging tasks often involve multiple contrasts, such as T1- and T2-weighted magnetic resonance imaging (MRI) data. These contrasts capture information associated with the same underlying anatomy and thus exhibit similarities. In this paper, we propose a Coupled Dictionary Learning based multi-contrast MRI reconstruction (CDLMRI) approach to leverage an available guidance contrast to restore the target contrast. Our approach consists of three stages: coupled dictionary learning, coupled sparse denoising, and $k$-space consistency enforcing. The first stage learns a group of dictionaries…
Segmentation of the left atrium (LA) is crucial for assessing its anatomy in both pre-operative atrial fibrillation (AF) ablation planning and post-operative follow-up studies. In this paper, we present a fully automated framework for left atrial segmentation in gadolinium-enhanced magnetic resonance images (GE-MRI) based on deep learning. We propose a fully convolutional neural network and explore the benefits of multi-task learning for performing both atrial segmentation and pre/post ablation classification. Our results show that, by sharing features between related tasks, the network can ga…
School of Psychology, The University of Queensland（昆士兰大学心理学院）；Queensland Brain Institute, The University of Queensland（昆士兰大学昆士兰脑研究所）；Centre for Advanced Imaging, The University of Queensland（昆士兰大学高级成像中心）
Whether it be in a man-made machine or a biological system, form and function are often directly related. In the latter, however, this particular relationship is often unclear due to the intricate nature of biology. Here we developed a geometric deep learning model capable of exploiting the actual structure of the cortex to learn the complex relationship between brain function and anatomy from structural and functional MRI data. Our model was not only able to predict the functional organization of human visual cortex from anatomical properties alone, but it was also able to predict nuanced var…
New Jersey Institute of Technology（新泽西理工学院）；Kessler Foundation（凯斯勒基金会）；Emory University（埃默里大学）；Atlanta VA Medical Center（亚特兰大退伍军人事务医疗中心）；Medical College of Wisconsin（威斯康星医学院）；Rutgers University - Newark（罗格斯大学纽瓦克分校）
We propose a fully 3D multi-path convolutional network to predict stroke lesions from 3D brain MRI images. Our multi-path model has independent encoders for different modalities containing residual convolutional blocks, weighted multi-path feature fusion from different modalities, and weighted fusion modules to combine encoder and decoder features. Compared to existing 3D CNNs like DeepMedic, 3D U-Net, and AnatomyNet, our networks achieves the highest statistically significant cross-validation accuracy of 60.5% on the large ATLAS benchmark of 220 patients. We also test our model on multi-modal…
