Learning Global and Local Features of Normal Brain Anatomy for Unsupervised Abnormality Detection
National Cancer Center Research Institute（国家癌症中心研究所）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Research Center for Advanced Science and Technology, The University of Tokyo（东京大学先进科学技术研究中心）；National Cancer Center Hospital（国家癌症中心医院）
In real-world clinical practice, overlooking unanticipated findings can result in serious consequences. However, supervised learning, which is the foundation for the current success of deep learning, only encourages models to identify abnormalities that are defined in datasets in advance. Therefore, abnormality detection must be implemented in medical images that are not limited to a specific disease category. In this study, we demonstrate an unsupervised learning framework for pixel-wise abnormality detection in brain magnetic resonance imaging captured from a patient population with metastatic brain tumor. Our concept is as follows: If an image reconstruction network can faithfully reproduce the global features of normal anatomy, then the abnormal lesions in unseen images can be identified based on the local difference from those reconstructed as normal by a discriminative network. Both networks are trained on a dataset comprising only normal images without labels. In addition, we devise a metric to evaluate the anatomical fidelity of the reconstructed images and confirm that the overall detection performance is improved when the image reconstruction network achieves a higher score. For evaluation, clinically significant abnormalities are comprehensively segmented. The results show that the area under the receiver operating characteristics curve values for metastatic brain tumors, extracranial metastatic tumors, postoperative cavities, and structural changes are 0.78, 0.61, 0.91, and 0.60, respectively.
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）
Creating a large-scale dataset of abnormality annotation on medical images is a labor-intensive and costly task. Leveraging weak supervision from readily available data such as radiology reports can compensate lack of large-scale data for anomaly detection methods. However, most of the current methods only use image-level pathological observations, failing to utilize the relevant anatomy mentions in reports. Furthermore, Natural Language Processing (NLP)-mined weak labels are noisy due to label sparsity and linguistic ambiguity. We propose an Anatomy-Guided chest X-ray Network (AGXNet) to addr…
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
University of Pittsburgh（匹兹堡大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cedars-Sinai Medical Center（西德萨斯医疗中心）；Mayo Clinic Arizona（梅奥诊所亚利桑那分部）
McGill University（麦吉尔大学）；MILA (Québec Artificial Intelligence Institute)（MILA（魁北克人工智能研究所））；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
We propose a hierarchically structured variational inference model for accurately disentangling observable evidence of disease (e.g. brain lesions or atrophy) from subject-specific anatomy in brain MRIs. With flexible, partially autoregressive priors, our model (1) addresses the subtle and fine-grained dependencies that typically exist between anatomical and pathological generating factors of an MRI to ensure the clinical validity of generated samples; (2) preserves and disentangles finer pathological details pertaining to a patient's disease state. Additionally, we experiment with an alternat…
We present GAAF, a Generalised Automatic Anatomy Finder, for the identification of generic anatomical locations in 3D CT scans. GAAF is an end-to-end pipeline, with dedicated modules for data pre-processing, model training, and inference. At it's core, GAAF uses a custom a localisation convolutional neural network (CNN). The CNN model is small, lightweight and can be adjusted to suit the particular application. The GAAF framework has so far been tested in the head and neck, and is able to find anatomical locations such as the centre-of-mass of the brainstem. GAAF was evaluated in an open-acces…
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Tencent AI Lab（腾讯人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
Recently, deep neural networks have greatly advanced undersampled Magnetic Resonance Image (MRI) reconstruction, wherein most studies follow the one-anatomy-one-network fashion, i.e., each expert network is trained and evaluated for a specific anatomy. Apart from inefficiency in training multiple independent models, such convention ignores the shared de-aliasing knowledge across various anatomies which can benefit each other. To explore the shared knowledge, one naive way is to combine all the data from various anatomies to train an all-round network. Unfortunately, despite the existence of th…
Full-Head Segmentation of MRI with Abnormal Brain Anatomy: Model and Data Release
The City College of New York（纽约市立学院）；New York University（纽约大学）；Georgetown University（乔治城大学）；University of North Carolina School of Medicine（北卡罗来纳大学医学院）；Soterix Medical
The University of Tokyo, Tokyo, Japan（日本 东京 东京大学）；KTH Royal Institute of Technology, Stockholm, Sweden（瑞典 斯德哥尔摩 KTH皇家理工学院）
Whole-brain surface extraction is an essential topic in medical imaging systems as it provides neurosurgeons with a broader view of surgical planning and abnormality detection. To solve the problem confronted in current deep learning skull stripping methods lacking prior shape information, we propose a new network architecture that incorporates knowledge of signed distance fields and introduce an additional Laplacian loss to ensure that the prediction results retain shape information. We validated our newly proposed method by conducting experiments on our brain magnetic resonance imaging datas…
Rethinking Convergence in Deep Learning: The Predictive-Corrective Paradigm for Anatomy-Informed Brain MRI Segmentation
School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院）；Key Laboratory of Signal Detection and Processing, Xinjiang University（新疆大学信号检测与处理重点实验室）；Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center（新疆多模态智能处理与信息安全工程技术研究中心）；National Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）
Unsupervised Anomaly Detection in Brain MRI via Disentangled Anatomy Learning
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China.（自动化与智能感知学院，上海交通大学，中国）；School of Computer Science, The University of Sydney, Sydney, Australia.（计算机科学学院，悉尼大学，澳大利亚）；Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.（放射科，仁济医院，医学院，上海交通大学，中国）；Biomedical Engineering Department, Tulane University, New Orleans, USA.（生物医学工程系， Tulane 大学，美国新奥尔良）
Contrastive Anatomy-Contrast Disentanglement: A Domain-General MRI Harmonization Method
Chair for AI for Image-Guided Diagnosis and Therapy, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany（人工智能辅助影像引导诊断与治疗研究所，慕尼黑技术大学（TUM）及慕尼黑技术大学医院，慕尼黑，德国）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心，慕尼黑，德国）；Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany（人工智能在医疗与健康领域研究所，慕尼黑技术大学（TUM）及慕尼黑技术大学医院，慕尼黑，德国）；Department of Radiation Oncology, TUM University Hospital, Munich, Germany（放射肿瘤科，慕尼黑技术大学医院，慕尼黑，德国）；Chair for Computer Vision and Artificial Intelligence, Technical University of Munich (TUM), Munich, Germany（计算机视觉与人工智能研究所，慕尼黑技术大学（TUM），慕尼黑，德国）；Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, CA, USA（斯坦福大学人工智能在医学与影像中心，斯坦福大学，斯坦福，CA，美国）；Deutsches Konsortium für Translationale Krebsforschung (DKTK), Partner Site Munich, Munich, Germany（德国转化癌症研究联盟（DKTK），慕尼黑合作伙伴站点，慕尼黑，德国）；Institute of Radiation Medicine (IRM), Department of Radiation Sciences (DRS), Helmholtz Center Munich, Munich, Germany（放射医学研究所（IRM），辐射科学部门（DRS），海德堡慕尼黑研究中心，慕尼黑，德国）
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（汉堡工业大学医疗技术与智能系统研究所）；jung diagnostics GmbH（jung diagnostics GmbH公司）；Hamburg University of Technology（汉堡工业大学）
Lesion detection in brain Magnetic Resonance Images (MRIs) remains a challenging task. MRIs are typically read and interpreted by domain experts, which is a tedious and time-consuming process. Recently, unsupervised anomaly detection (UAD) in brain MRI with deep learning has shown promising results to provide a quick, initial assessment. So far, these methods only rely on the visual appearance of healthy brain anatomy for anomaly detection. Another biomarker for abnormal brain development is the deviation between the brain age and the chronological age, which is unexplored in combination with …
Segmentation has been a major task in neuroimaging. A large number of automated methods have been developed for segmenting healthy and diseased brain tissues. In recent years, deep learning techniques have attracted a lot of attention as a result of their high accuracy in different segmentation problems. We present a new deep learning based segmentation method, DeepMRSeg, that can be applied in a generic way to a variety of segmentation tasks. The proposed architecture combines recent advances in the field of biomedical image segmentation and computer vision. We use a modified UNet architectur…
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for in…
Sathyabama University（萨蒂亚巴马大学）；Narasu’s Sarathy Institute of Technology（纳拉苏萨拉蒂技术学院）
Recognition of epileptic focal point is the important diagnosis when screening the epilepsy patients for latent surgical cures. The accurate localization is challenging one because of the low spatial resolution images with more noisy data. Positron Emission Tomography (PET) has now replaced the issues and caring a high resolution. This paper focuses the research of automated localization of epileptic seizures in brain functional images using symmetry based cluster approach. This approach presents a fully automated symmetry based brain abnormality detection method for PET sequences. PET images …
Enhancing Incomplete Multi-modal Brain Tumor Segmentation with Intra-modal Asymmetry and Inter-modal Dependency
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；School of Information Management, Jiangxi University of Finance and Economics（江西财经大学信息管理学院）；Institute for Infocomm Research, A*STAR（新加坡科技研究局信息通信研究所）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）
Probabilistic modelling has been an essential tool in medical image analysis, especially for analyzing brain Magnetic Resonance Images (MRI). Recent deep learning techniques for estimating high-dimensional distributions, in particular Variational Autoencoders (VAEs), opened up new avenues for probabilistic modeling. Modelling of volumetric data has remained a challenge, however, because constraints on available computation and training data make it difficult effectively leverage VAEs, which are well-developed for 2D images. We propose a method to model 3D MR brain volumes distribution by combi…
NVIDIA（英伟达）；DeepMind（深度思维）；Oxford Nanopore Technologies（牛津纳米孔技术公司）；University College London（伦敦大学学院）
Human anatomy, morphology, and associated diseases can be studied using medical imaging data. However, access to medical imaging data is restricted by governance and privacy concerns, data ownership, and the cost of acquisition, thus limiting our ability to understand the human body. A possible solution to this issue is the creation of a model able to learn and then generate synthetic images of the human body conditioned on specific characteristics of relevance (e.g., age, sex, and disease status). Deep generative models, in the form of neural networks, have been recently used to create synthe…
Lesion detection in brain Magnetic Resonance Images (MRI) remains a challenging task. State-of-the-art approaches are mostly based on supervised learning making use of large annotated datasets. Human beings, on the other hand, even non-experts, can detect most abnormal lesions after seeing a handful of healthy brain images. Replicating this capability of using prior information on the appearance of healthy brain structure to detect lesions can help computers achieve human level abnormality detection, specifically reducing the need for numerous labeled examples and bettering generalization of p…
A generalisable head MRI defacing pipeline: Evaluation on 2,566 meningioma scans
School of Biomedical Engineering & Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与成像科学学校）；Department of Neurosurgery, King’s College Hospital NHS Foundation Trust（伦敦国王学院医院神经外科部门）
ETH Zurich（苏黎世联邦理工学院）；Imperial College London（帝国理工学院）
Recent advances in deep learning led to novel generative modeling techniques that achieve unprecedented quality in generated samples and performance in learning complex distributions in imaging data. These new models in medical image computing have important applications that form clinically relevant and very challenging unsupervised learning problems. In this paper, we explore the feasibility of using state-of-the-art auto-encoder-based deep generative models, such as variational and adversarial auto-encoders, for one such task: abnormality detection in medical imaging. We utilize typical, pu…
This report describes my research activities in the Hasso Plattner Institute and summarizes my Ph.D. plan and several novels, end-to-end trainable approaches for analyzing medical images using deep learning algorithm. In this report, as an example, we explore different novel methods based on deep learning for brain abnormality detection, recognition, and segmentation. This report prepared for the doctoral consortium in the AIME-2017 conference.
Learning accurate rigid registration for longitudinal brain MRI from synthetic data
1 Division of Biomedical Imaging, KTH Royal Institute of Technology, Huddinge, Sweden 2 Athinoula A.\ Martinos Center for Biomedical Imaging, Charlestown, USA 3 Department of Radiology, Massachusetts General Hospital, Boston, USA 4 Department of Radiology, Harvard Medical School, Boston, USA 5 Computer Science \& Artificial Intelligence Laboratory, MIT, Cambridge, USA
ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（生物医学工程学院，生命科学与医学系，中国科学技术大学）；Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE) Lab, YRD-RIGHT, USTC Suzhou Institute for Advanced Research（医学影像、机器人、分析计算与学习（MIRACLE）实验室，YRD-RIGHT，中国科学技术大学苏州研究院）；Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology（江苏省多模态数字孪生技术重点实验室）；Biomedical Basic Research Center (BBRC) of Jiangsu Province（江苏省生物医学基础研究中心）；Department of Radiology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, USTC（放射科，中国科学技术大学第一附属医院，生命科学与医学系，中国科学技术大学）；Anhui IFLYTEK CO., Ltd（安徽科大讯飞股份有限公司）；School of Medicine, Stanford University（医学院，斯坦福大学）；State Key Laboratory of Precision and Intelligent Chemistry, Hefei, Anhui, China（安徽省精密与智能化学重点实验室，合肥，安徽，中国）
Anatomy-aware and acquisition-agnostic joint registration with SynthMorph
Athinoula A. Martinos Center for Biomedical Imaging（阿西努拉·A·马蒂诺斯生物医学成像中心）；Department of Radiology, Massachusetts General Hospital（麻省总医院放射科）；Department of Radiology, Harvard Medical School（哈佛医学院放射科）；Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）
University of Calcutta（加尔各答大学）；West Bengal University of Technology（西孟加拉技术大学）
Tumor segmentation from magnetic resonance imaging (MRI) data is an important but time consuming manual task performed by medical experts. Automating this process is a challenging task because of the high diversity in the appearance of tumor tissues among different patients and in many cases similarity with the normal tissues. MRI is an advanced medical imaging technique providing rich information about the human soft-tissue anatomy. There are different brain tumor detection and segmentation methods to detect and segment a brain tumor from MRI images. These detection and segmentation approache…
University of Central Florida（中佛罗里达大学）；Kyungpook National University（庆北国立大学）
Medical image segmentation being a substantial component of image processing plays a significant role to analyze gross anatomy, to locate an infirmity and to plan the surgical procedures. Segmentation of brain Magnetic Resonance Imaging (MRI) is of considerable importance for the accurate diagnosis. However, precise and accurate segmentation of brain MRI is a challenging task. Here, we present an efficient framework for segmentation of brain MR images. For this purpose, Gabor transform method is used to compute features of brain MRI. Then, these features are classified by using four different …
University of Copenhagen（哥本哈根大学）；Cerebriu A/S（Cerebriu公司）
Medical images used in clinical practice are heterogeneous and not the same quality as scans studied in academic research. Preprocessing breaks down in extreme cases when anatomy, artifacts, or imaging parameters are unusual or protocols are different. Methods robust to these variations are most needed. A novel deep learning method is proposed for fast and accurate segmentation of the human brain into 132 regions. The proposed model uses an efficient U-Net-like network and benefits from the intersection points of different views and hierarchical relations for the fusion of the orthogonal 2D pl…
Arizona State University（亚利桑那州立大学）；University of Pittsburgh（匹兹堡大学）；University of Southern California（南加州大学）
Applying network science approaches to investigate the functions and anatomy of the human brain is prevalent in modern medical imaging analysis. Due to the complex network topology, for an individual brain, mining a discriminative network representation from the multimodal brain networks is non-trivial. The recent success of deep learning techniques on graph-structured data suggests a new way to model the non-linear cross-modality relationship. However, current deep brain network methods either ignore the intrinsic graph topology or require a network basis shared within a group. To address the…
Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina at Chapel Hill（放射科与生物医学研究成像中心（BRIC）、北卡罗来纳大学教堂山分校）；Lampe Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University（Lampe联合生物医学工程部门、北卡罗来纳大学教堂山分校和北卡罗来纳州立大学）
New Jersey Institute of Technology（新泽西理工学院）；Kessler Foundation（凯斯勒基金会）；Emory University（埃默里大学）；Atlanta VA Medical Center（亚特兰大退伍军人事务医疗中心）；Medical College of Wisconsin（威斯康星医学院）；Rutgers University - Newark（罗格斯大学纽瓦克分校）
We propose a fully 3D multi-path convolutional network to predict stroke lesions from 3D brain MRI images. Our multi-path model has independent encoders for different modalities containing residual convolutional blocks, weighted multi-path feature fusion from different modalities, and weighted fusion modules to combine encoder and decoder features. Compared to existing 3D CNNs like DeepMedic, 3D U-Net, and AnatomyNet, our networks achieves the highest statistically significant cross-validation accuracy of 60.5% on the large ATLAS benchmark of 220 patients. We also test our model on multi-modal…
Philips Research（飞利浦研究院）；Skolkovo Institute of Science and Technology（斯科尔科沃科技学院）
Domain Adaptation (DA) methods are widely used in medical image segmentation tasks to tackle the problem of differently distributed train (source) and test (target) data. We consider the supervised DA task with a limited number of annotated samples from the target domain. It corresponds to one of the most relevant clinical setups: building a sufficiently accurate model on the minimum possible amount of annotated data. Existing methods mostly fine-tune specific layers of the pretrained Convolutional Neural Network (CNN). However, there is no consensus on which layers are better to fine-tune, e.…
This work is an endeavor to develop a deep learning methodology for automated anatomical labeling of a given region of interest (ROI) in brain computed tomography (CT) scans. We combine both local and global context to obtain a representation of the ROI. We then use Relation Networks (RNs) to predict the corresponding anatomy of the ROI based on its relationship score for each class. Further, we propose a novel strategy employing nearest neighbors approach for training RNs. We train RNs to learn the relationship of the target ROI with the joint representation of its nearest neighbors in each c…
TU Munich（慕尼黑工业大学）；Klinikum Rechts der Isar（Rechts der Isar 医院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Reliably modeling normality and differentiating abnormal appearances from normal cases is a very appealing approach for detecting pathologies in medical images. A plethora of such unsupervised anomaly detection approaches has been made in the medical domain, based on statistical methods, content-based retrieval, clustering and recently also deep learning. Previous approaches towards deep unsupervised anomaly detection model patches of normal anatomy with variants of Autoencoders or GANs, and detect anomalies either as outliers in the learned feature space or from large reconstruction errors. I…
Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRI
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（马萨诸塞州总医院阿西诺拉·A·马丁诺斯生物医学成像中心）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（马萨诸塞州总医院）
Surface-volume self-supervised representation learning of brain MRI for genetic discovery
The University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）；Yale School of Medicine（耶鲁大学医学院）；Texas A&M University（德克萨斯A&M大学）；Stanford University School of Medicine（斯坦福大学医学院）
Hamburg University of Technology（汉堡工业大学）；Jung Diagnostics GmbH（荣格诊断有限公司）
The use of supervised deep learning techniques to detect pathologies in brain MRI scans can be challenging due to the diversity of brain anatomy and the need for annotated data sets. An alternative approach is to use unsupervised anomaly detection, which only requires sample-level labels of healthy brains to create a reference representation. This reference representation can then be compared to unhealthy brain anatomy in a pixel-wise manner to identify abnormalities. To accomplish this, generative models are needed to create anatomically consistent MRI scans of healthy brains. While recent di…
Whiting School of Engineering, Johns Hopkins University（约翰斯·霍普金斯大学怀廷工程学院）；Rutgers, The State University of New Jersey（罗格斯新泽西州立大学）
Automatic segmentation of anatomical landmarks from ultrasound (US) plays an important role in the management of preterm neonates with a very low birth weight due to the increased risk of developing intraventricular hemorrhage (IVH) or other complications. One major problem in developing an automatic segmentation method for this task is the limited availability of annotated data. To tackle this issue, we propose a novel image synthesis method using multi-scale self attention generator to synthesize US images from various segmentation masks. We show that our method can synthesize high-quality U…
