Multimodal medical image fusion helps in combining contrasting features from two or more input imaging modalities to represent fused information in a single image. One of the pivotal clinical applications of medical image fusion is the merging of anatomical and functional modalities for fast diagnosis of malignant tissues. In this paper, we present a novel end-to-end unsupervised learning-based Convolutional Neural Network (CNN) for fusing the high and low frequency components of MRI-PET grayscale image pairs, publicly available at ADNI, by exploiting Structural Similarity Index (SSIM) as the loss function during training. We then apply color coding for the visualization of the fused image by quantifying the contribution of each input image in terms of the partial derivatives of the fused image. We find that our fusion and visualization approach results in better visual perception of the fused image, while also comparing favorably to previous methods when applying various quantitative assessment metrics.
Effective, robust, and automatic tools for brain tumor segmentation are needed for the extraction of information useful in treatment planning from magnetic resonance (MR) images. Context-aware artificial intelligence is an emerging concept for the development of deep learning applications for computer-aided medical image analysis. In this work, it is investigated whether the addition of contextual information from the brain anatomy in the form of white matter, gray matter, and cerebrospinal fluid masks and probability maps improves U-Net-based brain tumor segmentation. The BraTS2020 dataset wa…
Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIs
Stanford University（斯坦福大学）；University of Missouri（密苏里大学）；Missouri Institute of Mental Health（密苏里精神卫生研究所）；Weill Cornell Medicine（威尔康奈尔医学院）；SRI International（SRI国际研究所）
Anatomically Faithful Artifact Suppression in SENSE Accelerated Brain MRI
East China Normal University（华东师范大学）；Affiliated Hospital of Guizhou Medical University（贵州医科大学附属医院）；West China Hospital, Sichuan University（四川大学华西医院）；Chinese Academy of Medical Sciences（中国医学科学院）；RadioDynamic Medical（镭动医疗）；Lucile Packard Children’s Hospital（露西尔·帕卡德儿童医院）
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
University of Pittsburgh（匹兹堡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cedars-Sinai Medical Center（西德萨斯医疗中心）；Mayo Clinic Arizona（梅奥诊所亚利桑那分部）
Lausanne University Hospital and University of Lausanne（洛桑大学医院和洛桑大学）；Center for Biomedical Imaging（生物医学成像中心）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
A commonly adopted approach to carry out detection tasks in medical imaging is to rely on an initial segmentation. However, this approach strongly depends on voxel-wise annotations which are repetitive and time-consuming to draw for medical experts. An interesting alternative to voxel-wise masks are so-called "weak" labels: these can either be coarse or oversized annotations that are less precise, but noticeably faster to create. In this work, we address the task of brain aneurysm detection as a patch-wise binary classification with weak labels, in contrast to related studies that rather use s…
This work is an endeavor to develop a deep learning methodology for automated anatomical labeling of a given region of interest (ROI) in brain computed tomography (CT) scans. We combine both local and global context to obtain a representation of the ROI. We then use Relation Networks (RNs) to predict the corresponding anatomy of the ROI based on its relationship score for each class. Further, we propose a novel strategy employing nearest neighbors approach for training RNs. We train RNs to learn the relationship of the target ROI with the joint representation of its nearest neighbors in each c…
WASABI: A Metric for Evaluating Morphometric Plausibility of Synthetic Brain MRIs
Department of Radiology, Weill Cornell Medicine, New York, NY, USA（放射科，韦尔·科恩医学中心，纽约，纽约州，美国）；Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA（精神病学与行为科学系，斯坦福大学，斯坦福，加利福尼亚州，美国）；Department of Electrical and Computer Engineering, Cornell University, Ithaca, NY, USA（电气与计算机工程系，康奈尔大学，伊萨卡，纽约州，美国）
Evaluation of 3D Counterfactual Brain MRI Generation
Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA Department of Radiology, Stanford University, Stanford, CA, USA Dept. of Psychiatry \& Behavioral Sciences, Stanford University, Stanford, CA, USA Department of Computer Science, Stanford University, Stanford, CA, USA Department of Bioengineering, Stanford University, Stanford, CA, USA Department of Electrical Engineering, Stanford University, Stanford, CA, USA
Comparative Study of Anatomical and Learned Features in AI Models for Structural Brain MRI
Center for Data Science, New York University（纽约大学数据科学中心）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；Courant Institute of Mathematical Sciences, NYU（纽约大学柯朗数学科学研究所）
AA-ViT: Anatomically Aware Vision Transformer with Structural and Frequency Guidance for Contrast Enhanced Brain MRI Synthesis
Atlantic Technological University（大西洋理工大学）；Belfast Health and Social Care Trust（贝尔法斯特健康和社会关怀信托基金）；King’s College Hospital NHS Foundation Trust（国王学院医院国民保健服务信托基金）；King’s College London（伦敦国王学院）；Queen’s University Belfast（贝尔法斯特女王大学）
Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data
Center for Cognitive Neuroimaging（认知神经成像中心）；School of Psychology & Neuroscience（心理与神经科学学院）；University of Glasgow（格拉斯哥大学）；University of Brescia（布雷西亚大学）；Department of Information Engineering（信息工程系）
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH（VRVis虚拟现实与可视化研究中心有限公司）；Agfa NV（爱克发股份有限公司）；Antwerp University Hospital（安特卫普大学医院）；University of Antwerp（安特卫普大学）
To facilitate both the detection and the interpretation of findings in chest X-rays, comparison with a previous image of the same patient is very valuable to radiologists. Today, the most common approach for deep learning methods to automatically inspect chest X-rays disregards the patient history and classifies only single images as normal or abnormal. Nevertheless, several methods for assisting in the task of comparison through image registration have been proposed in the past. However, as we illustrate, they tend to miss specific types of pathological changes like cardiomegaly and effusion.…
Unsupervised Anomaly Detection in Brain MRI via Disentangled Anatomy Learning
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China.（自动化与智能感知学院，上海交通大学，中国）；School of Computer Science, The University of Sydney, Sydney, Australia.（计算机科学学院，悉尼大学，澳大利亚）；Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.（放射科，仁济医院，医学院，上海交通大学，中国）；Biomedical Engineering Department, Tulane University, New Orleans, USA.（生物医学工程系， Tulane 大学，美国新奥尔良）
Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRI
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（马萨诸塞州总医院阿西诺拉·A·马丁诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（马萨诸塞州总医院）
TractGraphFormer: Anatomically Informed Hybrid Graph CNN-Transformer Network for Interpretable Sex and Age Prediction from Diffusion MRI Tractography
Brigham and Women's Hospital, Harvard Medical School（哈佛医学院布莱根妇女医院）；University of Electronic Science and Technology of China（电子科技大学）；Beth Israel Deaconess Medical Center, Harvard Medical School（哈佛医学院贝斯以色列女执事医疗中心）；The University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Boston University School of Medicine（波士顿大学医学院）；Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院）
Accurate detection of anatomical landmarks is an essential step in several medical imaging tasks. We propose a novel communicative multi-agent reinforcement learning (C-MARL) system to automatically detect landmarks in 3D brain images. C-MARL enables the agents to learn explicit communication channels, as well as implicit communication signals by sharing certain weights of the architecture among all the agents. The proposed approach is evaluated on two brain imaging datasets from adult magnetic resonance imaging (MRI) and fetal ultrasound scans. Our experiments show that involving multiple coo…
SAME++: A Self-supervised Anatomical eMbeddings Enhanced medical image registration framework using stable sampling and regularized transformation
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Alibaba Group（阿里巴巴集团）；Johns Hopkins University（约翰斯·霍普金斯大学）；The First Affiliated Hospital of Zhejiang University（浙江大学附属第一医院）
Accuracy segmentation of brain structures could be helpful for glioma and radiotherapy planning. However, due to the visual and anatomical differences between different modalities, the accurate segmentation of brain structures becomes challenging. To address this problem, we first construct a residual block based U-shape network with a deep encoder and shallow decoder, which can trade off the framework performance and efficiency. Then, we introduce the Tversky loss to address the issue of the class imbalance between different foreground and the background classes. Finally, a model ensemble str…
Segmentation of brain structures from magnetic resonance (MR) scans plays an important role in the quantification of brain morphology. Since 3D deep learning models suffer from high computational cost, 2D deep learning methods are favored for their computational efficiency. However, existing 2D deep learning methods are not equipped to effectively capture 3D spatial contextual information that is needed to achieve accurate brain structure segmentation. In order to overcome this limitation, we develop an Anatomical Context-Encoding Network (ACEnet) to incorporate 3D spatial and anatomical conte…
Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalities
University of Science and Technology of China（中国科学技术大学）；Tianjin University（天津大学）；Beijing Tiantan Hospital, Capital Medical University（首都医科大学附属北京天坛医院）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）
The 4D Human Embryonic Brain Atlas: spatiotemporal atlas generation for rapid anatomical changes
Erasmus MC（埃拉斯姆斯医学院）；University Medical Center Groningen（格罗宁根大学医学中心）；Amsterdam UMC location University of Amsterdam（阿姆斯特丹大学阿姆斯特丹医学中心分部）；Amsterdam Reproduction and Development research institute（阿姆斯特丹生殖与发育研究所）
Changes over time in brain anatomy can provide important insight for treatment design or scientific analyses. We present a method that predicts how a brain MRI for an individual will change over time. We model changes using a diffeomorphic deformation field that we predict using function using convolutional neural networks. Given a predicted deformation field, a baseline scan can be warped to give a prediction of the brain scan at a future time. We demonstrate the method using the ADNI cohort, and analyze how performance is affected by model variants and the subject-specific information provid…
University of Oxford（牛津大学）；Leiden University Medical Center（莱顿大学医学中心）；Shanghai Jiao Tong University（上海交通大学）
Two-dimensional (2D) freehand ultrasound is the mainstay in prenatal care and fetal growth monitoring. The task of matching corresponding cross-sectional planes in the 3D anatomy for a given 2D ultrasound brain scan is essential in freehand scanning, but challenging. We propose AdLocUI, a framework that Adaptively Localizes 2D Ultrasound Images in the 3D anatomical atlas without using any external tracking sensor.. We first train a convolutional neural network with 2D slices sampled from co-aligned 3D ultrasound volumes to predict their locations in the 3D anatomical atlas. Next, we fine-tune …
Center for Biomedical Imaging, CIBM（生物医学成像中心（CIBM））
Brain aneurysm detection in Time-Of-Flight Magnetic Resonance Angiography (TOF-MRA) has undergone drastic improvements with the advent of Deep Learning (DL). However, performances of supervised DL models heavily rely on the quantity of labeled samples, which are extremely costly to obtain. Here, we present a DL model for aneurysm detection that overcomes the issue with ''weak'' labels: oversized annotations which are considerably faster to create. Our weak labels resulted to be four times faster to generate than their voxel-wise counterparts. In addition, our model leverages prior anatomical k…
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this paper we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end…
Brain Imaging Data Structure (BIDS) allows the user to organise brain imaging data into a clear and easy standard directory structure. BIDS is widely supported by the scientific community and is considered a powerful standard for management. The original BIDS is limited to images or data related to the brain. Medical Imaging Data Structure (MIDS) was therefore conceived with the objective of extending this methodology to other anatomical regions and other types of imaging systems in these areas.
Fine-Tuning TransMorph with Gradient Correlation for Anatomical Alignment
University Hospital Augsburg（奥格斯堡大学医院）；Technical University of Munich（慕尼黑工业大学）；Institute of Digital Medicine, University Hospital Augsburg（奥格斯堡大学医院数字医学研究所）
KTH Royal Institute of Technology（瑞典皇家理工学院）；Karolinska Institute（卡罗林斯卡学院）；Karolinska University Hospital（卡罗林斯卡大学医院）；University Fernando Pessoa Canarias（费尔南多·佩索阿加那利群岛大学）；King’s College London（伦敦国王学院）
Analyzing and predicting brain aging is essential for early prognosis and accurate diagnosis of cognitive diseases. The technique of neuroimaging, such as Magnetic Resonance Imaging (MRI), provides a noninvasive means of observing the aging process within the brain. With longitudinal image data collection, data-intensive Artificial Intelligence (AI) algorithms have been used to examine brain aging. However, existing state-of-the-art algorithms tend to be restricted to group-level predictions and suffer from unreal predictions. This paper proposes a methodology for generating longitudinal MRI s…
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Renmin University of China（中国人民大学）；Shenzhen University（深圳大学）
Magnetic resonance imaging plays an important role in computer-aided diagnosis and brain exploration. However, limited by hardware, scanning time and cost, it's challenging to acquire high-resolution (HR) magnetic resonance (MR) image clinically. In this paper, fine perceptive generative adversarial networks (FP-GANs) is proposed to produce HR MR images from low-resolution counterparts. It can cope with the detail insensitive problem of the existing super-resolution model in a divide-and-conquer manner. Specifically, FP-GANs firstly divides an MR image into low-frequency global approximation a…
Institute for Biomedical Engineering, University and ETH Zurich（苏黎世大学与苏黎世联邦理工学院生物医学工程研究所）
Purpose: To implement a brain segmentation pipeline based on convolutional neural networks, which rapidly segments 3D volumes into 27 anatomical structures. To provide an extensive, comparative study of segmentation performance on various contrasts of magnetic resonance imaging (MRI) and computed tomography (CT) scans. Methods: Deep convolutional neural networks are trained to segment 3D MRI (MPRAGE, DWI, FLAIR) and CT scans. A large database of in total 851 MRI/CT scans is used for neural network training. Training labels are obtained on the MPRAGE contrast and coregistered to the other imagi…
Enhancing Alzheimer's Diagnosis: Leveraging Anatomical Landmarks in Graph Convolutional Neural Networks on Tetrahedral Meshes
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA（计算与增强智能学院，亚利桑那州立大学）；Amazon AGI, Redmond, WA, USA（亚马逊人工智慧部门）；Banner Alzheimer’s Institute, Phoenix, AZ, USA（贝纳尔德阿尔茨海默病研究所）
South China University of Technology（华南理工大学）；University of Utah（犹他大学）
In medical applications, the same anatomical structures may be observed in multiple modalities despite the different image characteristics. Currently, most deep models for multimodal segmentation rely on paired registered images. However, multimodal paired registered images are difficult to obtain in many cases. Therefore, developing a model that can segment the target objects from different modalities with unpaired images is significant for many clinical applications. In this work, we propose a novel two-stream translation and segmentation unified attentional generative adversarial network (U…
University of Mons（蒙斯大学）；Fund for Scientific Research - FNRS (F.R.S.-FNRS)（比利时科学研究基金）；Swiss Data Science Center, EPFL and ETH Zürich（瑞士数据科学中心，洛桑联邦理工学院和苏黎世联邦理工学院）
While the prevalence of Autism Spectrum Disorder (ASD) is increasing, research continues in an effort to identify common etiological and pathophysiological bases. In this regard, modern machine learning and network science pave the way for a better understanding of the neuropathology and the development of diagnosis aid systems. The present work addresses the classification of neurotypical and ASD subjects by combining knowledge about both the structure and the functional activity of the brain. In particular, we model the brain structure as a graph, and the resting-state functional MRI (rs-fMR…
Automatic and reliable quantitative tools for MR brain image analysis are a very valuable resources for both clinical and research environments. In the last years, this field has experienced many advances with successful techniques based on label fusion and more recently deep learning. However, few of them have been specifically designed to provide a dense anatomical labelling at multiscale level and to deal with brain anatomical alterations such as white matter lesions. In this work, we present a fully automatic pipeline (vol2Brain) for whole brain segmentation and analysis which densely labe…
Lausanne University Hospital and University of Lausanne（洛桑大学医院和洛桑大学）；Siemens Healthcare AG（西门子医疗股份公司）；Center for Biomedical Imaging (CIBM)（生物医学成像中心）
Purpose: Whole-heart MRA techniques typically target pre-determined motion states and address cardiac and respiratory dynamics independently. We propose a novel fast reconstruction algorithm, applicable to ungated free-running sequences, that leverages inherent similarities in the acquired data to avoid such physiological constraints. Theory and Methods: The proposed SIMilarity-Based Angiography (SIMBA) method clusters the continuously acquired k-space data in order to find a motion-consistent subset that can be reconstructed into a motion-suppressed whole-heart MRA. Free-running 3D radial dat…
