Brain-computer interfaces (BCIs) are enabling a range of new possibilities and routes for augmenting human capability. Here, we propose BCIs as a route towards forms of computation, i.e. computational imaging, that blend the brain with external silicon processing. We demonstrate ghost imaging of a hidden scene using the human visual system that is combined with an adaptive computational imaging scheme. This is achieved through a projection pattern `carving' technique that relies on real-time feedback from the brain to modify patterns at the light projector, thus enabling more efficient and higher resolution imaging. This brain-computer connectivity demonstrates a form of augmented human computation that could in the future extend the sensing range of human vision and provide new approaches to the study of the neurophysics of human perception. As an example, we illustrate a simple experiment whereby image reconstruction quality is affected by simultaneous conscious processing and readout of the perceived light intensities.
ShanghaiTech University（上海科技大学）；University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）
Multi-spectral optoacoustic tomography (MSOT) is an emerging optical imaging method providing multiplex molecular and functional information from the rodent brain. It can be greatly augmented by magnetic resonance imaging (MRI) that offers excellent soft-tissue contrast and high-resolution brain anatomy. Nevertheless, registration of multi-modal images remains challenging, chiefly due to the entirely different image contrast rendered by these modalities. Previously reported registration algorithms mostly relied on manual user-dependent brain segmentation, which compromised data interpretation …
School of Biomedical Engineering（生物医学工程学院）；Universidad Autónoma de Ciudad Juárez（华雷斯自治大学）
The brain is a complex organ controlling cognitive process and physical functions. Tumors in the brain are accelerated cell growths affecting the normal function and processes in the brain. MRI scans provides detailed images of the body being one of the most common tests to diagnose brain tumors. The process of segmentation of brain tumors from magnetic resonance imaging can provide a valuable guide for diagnosis, treatment planning and prediction of results. Here we consider the problem brain tumor segmentation using a Deep learning architecture for use in tumor segmentation. Although the pro…
Singapore University of Technology and Design（新加坡科技设计大学）
Brain imaging data such as EEG or MEG are high-dimensional spatiotemporal data often degraded by complex, non-Gaussian noise. For reliable analysis of brain imaging data, it is important to extract discriminative, low-dimensional intrinsic representation of the recorded data. This work proposes a new method to learn the low-dimensional representations from the noise-degraded measurements. In particular, our work proposes a new deep neural network design that integrates graph information such as brain connectivity with fully-connected layers. Our work leverages efficient graph filter design usi…
Singapore University of Technology and Design（新加坡科技设计大学）
We propose an algorithm to uncover the intrinsic low-rank component of a high-dimensional, graph-smooth and grossly-corrupted dataset, under the situations that the underlying graph is unknown. Based on a model with a low-rank component plus a sparse perturbation, and an initial graph estimation, our proposed algorithm simultaneously learns the low-rank component and refines the graph. Our evaluations using synthetic and real brain imaging data in unsupervised and supervised classification tasks demonstrate encouraging performance.
Clinical diagnostic and treatment decisions rely upon the integration of patient-specific data with clinical reasoning. Cancer presents a unique context that influence treatment decisions, given its diverse forms of disease evolution. Biomedical imaging allows noninvasive assessment of disease based on visual evaluations leading to better clinical outcome prediction and therapeutic planning. Early methods of brain cancer characterization predominantly relied upon statistical modeling of neuroimaging data. Driven by the breakthroughs in computer vision, deep learning became the de facto standar…
Northeastern University（东北大学）；Harvard Medical School（哈佛医学院）
Brain extraction or whole brain segmentation is an important first step in many of the neuroimage analysis pipelines. The accuracy and robustness of brain extraction, therefore, is crucial for the accuracy of the entire brain analysis process. With the aim of designing a learning-based, geometry-independent and registration-free brain extraction tool in this study, we present a technique based on an auto-context convolutional neural network (CNN), in which intrinsic local and global image features are learned through 2D patches of different window sizes. In this architecture three parallel 2D …
Brain signals constitute the information that are processed by millions of brain neurons (nerve cells and brain cells). These brain signals can be recorded and analyzed using various of non-invasive techniques such as the Electroencephalograph (EEG), Magneto-encephalograph (MEG) as well as brain-imaging techniques such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and others, which will be discussed briefly in this paper. This paper discusses about the currently emerging techniques such as the usage of different Deep Learning (DL) algorithms for the analysis of these brain sign…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Whole brain segmentation on structural magnetic resonance imaging (MRI) is essential for understanding neuroanatomical-functional relationships. Traditionally, multi-atlas segmentation has been regarded as the standard method for whole brain segmentation. In past few years, deep convolutional neural network (DCNN) segmentation methods have demonstrated their advantages in both accuracy and computational efficiency. Recently, we proposed the spatially localized atlas network tiles (SLANT) method, which is able to segment a 3D MRI brain scan into 132 anatomical regions. Commonly, DCNN segmentati…
OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks
Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen Memorial Hospital, Sun Yat-sen University（中山大学孙逸仙纪念医院）；Southern Medical University（南方医科大学）；Zhongshan Hospital, Fudan University（复旦大学中山医院）；Department of Neurosurgery, Prince of Wales Hospital（威尔士王室医院神经外科部）
City University of London（伦敦城市大学）；Royal Brompton Hospital（皇家布朗普顿医院）；Imperial College London（帝国理工学院）
Magnetic Resonance Spectroscopy (MRS) provides valuable information to help with the identification and understanding of brain tumors, yet MRS is not a widely available medical imaging modality. Aiming to counter this issue, this research draws on the advancements in machine learning techniques in other fields for the generation of artificial data. The generated methods were tested through the evaluation of their output against that of a real-world labelled MRS brain tumor data-set. Furthermore the resultant output from the generative techniques were each used to train separate traditional cla…
University of Pennsylvania（宾夕法尼亚大学）；Columbia University（哥伦比亚大学）；Ludwig-Maximilian University（路德维希-马克西米利安大学）；University of Birmingham（伯明翰大学）
The imaging community has increasingly adopted machine learning (ML) methods to provide individualized imaging signatures related to disease diagnosis, prognosis, and response to treatment. Clinical neuroscience and cancer imaging have been two areas in which ML has offered particular promise. However, many neurologic and neuropsychiatric diseases, as well as cancer, are often heterogeneous in terms of their clinical manifestations, neuroanatomical patterns or genetic underpinnings. Therefore, in such cases, seeking a single disease signature might be ineffectual in delivering individualized p…
Accurate detection of anatomical landmarks is an essential step in several medical imaging tasks. We propose a novel communicative multi-agent reinforcement learning (C-MARL) system to automatically detect landmarks in 3D brain images. C-MARL enables the agents to learn explicit communication channels, as well as implicit communication signals by sharing certain weights of the architecture among all the agents. The proposed approach is evaluated on two brain imaging datasets from adult magnetic resonance imaging (MRI) and fetal ultrasound scans. Our experiments show that involving multiple coo…
Imperial College London（伦敦帝国理工学院）；Academic Medical Center（学术医学中心）
Machine learning analysis of neuroimaging data can accurately predict chronological age in healthy people and deviations from healthy brain ageing have been associated with cognitive impairment and disease. Here we sought to further establish the credentials of "brain-predicted age" as a biomarker of individual differences in the brain ageing process, using a predictive modelling approach based on deep learning, and specifically convolutional neural networks (CNN), and applied to both pre-processed and raw T1-weighted MRI data. Firstly, we aimed to demonstrate the accuracy of CNN brain-predict…
ConnectomeDiffuser: Generative AI Enables Brain Network Construction from Diffusion Tensor Imaging
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）；Department of Mathematics, Hong Kong Baptist University（香港 Baptist大学数学系）
Lehigh University（里海大学）；University of Pennsylvania（宾夕法尼亚大学）
Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies. In this paper, we propose an interpretable Graph Convolutional Network (GCN) framework for the identification and classification of Alzheimer's disease (AD) using multi-modality brain imaging data. Specifically, we extended the Gradient Class Activation Mapping (Grad-CAM) technique to quantify the most discriminative features identified by GCN from brain connectivity patterns. We then utilized them to find signature regions of interest (ROIs) by detecting…
University of Sydney（悉尼大学）；Sydney Neuroimaging Analysis Centre（悉尼神经影像分析中心）
Diffusion Weighted Imaging (DWI) is an advanced imaging technique commonly used in neuroscience and neurological clinical research through a Diffusion Tensor Imaging (DTI) model. Volumetric scalar metrics including fractional anisotropy, mean diffusivity, and axial diffusivity can be derived from the DTI model to summarise water diffusivity and other quantitative microstructural information for clinical studies. However, clinical practice constraints can lead to sub-optimal DWI acquisitions with missing slices (either due to a limited field of view or the acquisition of disrupted slices). To a…
Chulalongkorn University（朱拉隆功大学）；European University of Bangladesh（孟加拉国欧洲大学）；University of Liberal Arts Bangladesh（孟加拉国文科大学）
The growth of abnormal cells in the brain's tissue causes brain tumors. Brain tumors are considered one of the most dangerous disorders in children and adults. It develops quickly, and the patient's survival prospects are slim if not appropriately treated. Proper treatment planning and precise diagnoses are essential to improving a patient's life expectancy. Brain tumors are mainly diagnosed using magnetic resonance imaging (MRI). As part of a convolution neural network (CNN)-based illustration, an architecture containing five convolution layers, five max-pooling layers, a Flatten layer, and t…
University of Girona（赫罗纳大学）；Computer Vision and Robotics Institute（计算机视觉与机器人研究所）
In recent years, deep convolutional neural networks (CNNs) have shown record-shattering performance in a variety of computer vision problems, such as visual object recognition, detection and segmentation. These methods have also been utilised in medical image analysis domain for lesion segmentation, anatomical segmentation and classification. We present an extensive literature review of CNN techniques applied in brain magnetic resonance imaging (MRI) analysis, focusing on the architectures, pre-processing, data-preparation and post-processing strategies available in these works. The aim of thi…
BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification
College of Computing and Data Science, Nanyang Technological University, Singapore（南洋理工大学计算与数据科学学院）；Saw Swee Hock School of Public Health at National University of Singapore, Singapore（新加坡国立大学 Saw Swee Hock 公共卫生学院）；Xi’an Jiaotong University, Xi’an, Shaanxi（西安交通大学）；S-Lab, Nanyang Technological University, Singapore（南洋理工大学 S 实验室）
A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts
Imperial College London（帝国理工学院）；UK Dementia Research Institute（英国痴呆症研究所）；Beijing Institute of Technology（北京理工大学）；Capital Medical University（首都医科大学）；Beijing Tiantan Hospital（北京天坛医院）
Studying the Effects of Sex-related Differences on Brain Age Prediction using brain MR Imaging
University of Calgary（卡尔加里大学）；Indian Institute of Technology Kharagpur（印度理工学院卡拉格布尔分校）；Dayalbagh Educational Institute (Deemed University)（达亚尔巴格教育学院（被视为大学））
We describe Countersynth, a conditional generative model of diffeomorphic deformations that induce label-driven, biologically plausible changes in volumetric brain images. The model is intended to synthesise counterfactual training data augmentations for downstream discriminative modelling tasks where fidelity is limited by data imbalance, distributional instability, confounding, or underspecification, and exhibits inequitable performance across distinct subpopulations. Focusing on demographic attributes, we evaluate the quality of synthesized counterfactuals with voxel-based morphometry, clas…
Symmetry Awareness Encoded Deep Learning Framework for Brain Imaging Analysis
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；Brain and Mind Centre, University of Sydney（悉尼大学脑与心智中心）；Sydney Neuroimaging Analysis Centre（悉尼神经影像分析中心）；School of Mathematics and Statistics, University of Sydney（悉尼大学数学与统计学院）；I-MED Radiology Network（I-MED放射学网络）
National Institute of Mental Health（美国国家精神卫生研究所）；Birkbeck College（伯克贝克学院）；University College London（伦敦大学学院）
Deep neural networks have brought remarkable breakthroughs in medical image analysis. However, due to their data-hungry nature, the modest dataset sizes in medical imaging projects might be hindering their full potential. Generating synthetic data provides a promising alternative, allowing to complement training datasets and conducting medical image research at a larger scale. Diffusion models recently have caught the attention of the computer vision community by producing photorealistic synthetic images. In this study, we explore using Latent Diffusion Models to generate synthetic images from…
Silesian University in Opava（奥帕瓦西里西亚大学）；University of Oxford（牛津大学）；Ostrava University（俄斯特拉发大学）
Matching MRI brain images between patients or mapping patients' MRI slices to the simulated atlas of a brain is key to the automatic registration of MRI of a brain. The ability to match MRI images would also enable such applications as indexing and searching MRI images among multiple patients or selecting images from the region of interest. In this work, we have introduced robustness, accuracy and cumulative distance metrics and methodology that allows us to compare different techniques and approaches in matching brain MRI of different patients or matching MRI brain slice to a position in the …
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Northwestern Polytechnical University（西北工业大学）；University of Pittsburgh（匹兹堡大学）；University of Sydney（悉尼大学）；JD Finance America Corporation（京东金融美国公司）
In our previous work, $i.e.$, HNF-Net, high-resolution feature representation and light-weight non-local self-attention mechanism are exploited for brain tumor segmentation using multi-modal MR imaging. In this paper, we extend our HNF-Net to HNF-Netv2 by adding inter-scale and intra-scale semantic discrimination enhancing blocks to further exploit global semantic discrimination for the obtained high-resolution features. We trained and evaluated our HNF-Netv2 on the multi-modal Brain Tumor Segmentation Challenge (BraTS) 2021 dataset. The result on the test set shows that our HNF-Netv2 achieved…
Center for Biomedical Image Computing and Analytics（生物医学图像计算与分析中心）；Perelman School of Medicine（佩雷尔曼医学院）；University of Pennsylvania（宾夕法尼亚大学）
Decoding brain functional states underlying different cognitive processes using multivariate pattern recognition techniques has attracted increasing interests in brain imaging studies. Promising performance has been achieved using brain functional connectivity or brain activation signatures for a variety of brain decoding tasks. However, most of existing studies have built decoding models upon features extracted from imaging data at individual time points or temporal windows with a fixed interval, which might not be optimal across different cognitive processes due to varying temporal durations…
A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland（东芬兰大学A.I.维尔塔宁分子科学研究所）
Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In MRI, transfer learning is important for developing strategies that address the variation in MR images. Additionally, transfer learning is beneficial to re-utilize machine learning models that were trained to solve related tasks to the task of interest. Our goal is to identify research directions, gaps of knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging. We perform…
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Northwestern Polytechnical University（西北工业大学）；University of Pittsburgh（匹兹堡大学）；University of Sydney（悉尼大学）；JD Finance America Corporation（京东金融美国公司）
In this paper, we propose a Hybrid High-resolution and Non-local Feature Network (H2NF-Net) to segment brain tumor in multimodal MR images. Our H2NF-Net uses the single and cascaded HNF-Nets to segment different brain tumor sub-regions and combines the predictions together as the final segmentation. We trained and evaluated our model on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 dataset. The results on the test set show that the combination of the single and cascaded models achieved average Dice scores of 0.78751, 0.91290, and 0.85461, as well as Hausdorff distances ($95\%$…
Advanced Brain Tissue Imaging with Data-Consistent Diffusion Priors in Laminographic X-Ray Nanoimaging
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Paul Scherrer Institute（保罗·谢勒研究所）；The Francis Crick Institute（弗朗西斯·克里克研究所）；Mineral Resource CSIRO（澳大利亚联邦科学与工业研究组织矿产资源部）；ESRF, The European Synchrotron（欧洲同步辐射光源）
Inria（法国国家信息与自动化研究所）；CEA（法国原子能与替代能源委员会）；Univ. Paris Saclay（巴黎萨克雷大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立理工学院）
Cognitive brain imaging is accumulating datasets about the neural substrate of many different mental processes. Yet, most studies are based on few subjects and have low statistical power. Analyzing data across studies could bring more statistical power; yet the current brain-imaging analytic framework cannot be used at scale as it requires casting all cognitive tasks in a unified theoretical framework. We introduce a new methodology to analyze brain responses across tasks without a joint model of the psychological processes. The method boosts statistical power in small studies with specific co…
Image modality recognition is essential for efficient imaging workflows in current clinical environments, where multiple imaging modalities are used to better comprehend complex diseases. Emerging biomarkers from novel, rare modalities are being developed to aid in such understanding, however the availability of these images is often limited. This scenario raises the necessity of recognising new imaging modalities without them being collected and annotated in large amounts. In this work, we present a few-shot learning model for limited training examples based on Deep Triplet Networks. We show …
Lost in Distortion: Uncovering the Domain Gap Between Computer Vision and Brain Imaging -- A Study on Pretraining for Age Prediction
Center for Translational Research in Neuroimaging and Data Science（转化神经影像与数据科学研究中心）；College of Artificial Intelligence, Chengdu University of Information Technology（成都信息科技大学人工智能学院）；Department of Applied Mathematics and Statistics（应用数学与统计学系）；McWilliams School of Biomedical Informatics（麦卡威廉生物医学信息学学院）；Department of Applied Mathematics and Theoretical Physics（应用数学与理论物理系）；School of Biomedical Engineering and Imaging Sciences（生物医学工程与影像科学学院）
A large-scale heterogeneous 3D magnetic resonance brain imaging dataset for self-supervised learning
Department of Computer Science, University of Copenhagen（计算机科学系，哥本哈根大学）；Pioneer Centre For AI（人工智能先锋中心）；Copenhagen Research Centre for Biological and Precision Psychiatry（哥本哈根生物与精准精神病学研究中心）；Mental Health Centre Copenhagen（哥本哈根心理健康中心）；Copenhagen University Hospital（哥本哈根大学医院）；Rigshospitalet（皇家医院）；Radiological AI Testcenter（放射学AI测试中心）；Athinoula A. Martinos Center for Biomedical Imaging（Athinoula A. Martinos生物医学成像中心）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Johns Hopkins University（约翰霍普金斯大学）
University of Girona（赫罗纳大学）；Computer Vision and Robotics Institute（计算机视觉与机器人研究所）
Accurate brain tissue segmentation in Magnetic Resonance Imaging (MRI) has attracted the attention of medical doctors and researchers since variations in tissue volume help in diagnosing and monitoring neurological diseases. Several proposals have been designed throughout the years comprising conventional machine learning strategies as well as convolutional neural networks (CNN) approaches. In particular, in this paper, we analyse a sub-group of deep learning methods producing dense predictions. This branch, referred in the literature as Fully CNN (FCNN), is of interest as these architectures …
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Yale University（耶鲁大学）；University of Iowa（爱荷华大学）；Augusta University（奥古斯塔大学）；University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；Johns Hopkins University（约翰斯·霍普金斯大学）
Rodent stroke models are important for evaluating treatments and understanding the pathophysiology and behavioral changes of brain ischemia, and magnetic resonance imaging (MRI) is a valuable tool for measuring outcome in preclinical studies. Brain extraction is an essential first step in most neuroimaging pipelines; however, it can be challenging in the presence of severe pathology and when dataset quality is highly variable. Convolutional neural networks (CNNs) can improve accuracy and reduce operator time, facilitating high throughput preclinical studies. As part of an ongoing preclinical s…
AI generated annotations for Breast, Brain, Liver, Lungs and Prostate cancer collections in National Cancer Institute Imaging Data Commons
BAMF Health（BAMF健康公司）；National Institute of Health（美国国立卫生研究院）；Frederick National Laboratory for Cancer Research（弗雷德里克国家癌症研究实验室）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
Accurate detection of brain tumors could save lots of lives and increasing the accuracy of this binary classification even as much as a few percent has high importance. Neural Gas Networks (NGN) is a fast, unsupervised algorithm that could be used in data clustering, image pattern recognition, and image segmentation. In this research, we used the metaheuristic Firefly Algorithm (FA) for image contrast enhancement as pre-processing and NGN weights for feature extraction and segmentation of Magnetic Resonance Imaging (MRI) data on two brain tumor datasets from the Kaggle platform. Also, tumor cl…
Image segmentation some of the challenging issues on brain magnetic resonance image tumor segmentation caused by the weak correlation between magnetic resonance imaging intensity and anatomical meaning.With the objective of utilizing more meaningful information to improve brain tumor segmentation,an approach which employs bilateral symmetry information as an additional feature for segmentation is proposed.This is motivated by potential performance improvement in the general automatic brain tumor segmentation systems which are important for many medical and scientific applications.Brain Magneti…
Mayo Clinic（梅奥诊所）；Moffitt Cancer Center and Research Institute（莫菲特癌症中心与研究所）
Whole brain extraction, also known as skull stripping, is a process in neuroimaging in which non-brain tissue such as skull, eyeballs, skin, etc. are removed from neuroimages. Skull striping is a preliminary step in presurgical planning, cortical reconstruction, and automatic tumor segmentation. Despite a plethora of skull stripping approaches in the literature, few are sufficiently accurate for processing pathology-presenting MRIs, especially MRIs with brain tumors. In this work we propose a deep learning approach for skull striping common MRI sequences in oncology such as T1-weighted with ga…
Medical image segmentation has become an essential technique in clinical and research-oriented applications. Because manual segmentation methods are tedious, and fully automatic segmentation lacks the flexibility of human intervention or correction, semi-automatic methods have become the preferred type of medical image segmentation. We present a hybrid, semi-automatic segmentation method in 3D that integrates both region-based and boundary-based procedures. Our method differs from previous hybrid methods in that we perform region-based and boundary-based approaches separately, which allows for…
University of California, San Francisco（加州大学旧金山分校）
Positron Emission Tomography (PET) is a functional imaging modality widely used in neuroscience studies. To obtain meaningful quantitative results from PET images, attenuation correction is necessary during image reconstruction. For PET/MR hybrid systems, PET attenuation is challenging as Magnetic Resonance (MR) images do not reflect attenuation coefficients directly. To address this issue, we present deep neural network methods to derive the continuous attenuation coefficients for brain PET imaging from MR images. With only Dixon MR images as the network input, the existing U-net structure wa…
