Our ability to generalize beyond training data to novel, out-of-distribution, image degradations is a hallmark of primate vision. The predictive brain, exemplified by predictive coding networks (PCNs), has become a prominent neuroscience theory of neural computation. Motivated by the recent successes of variational autoencoders (VAEs) in machine learning, we rigorously derive a correspondence between PCNs and VAEs. This motivates us to consider iterative extensions of VAEs (iVAEs) as plausible variational extensions of the PCNs. We further demonstrate that iVAEs generalize to distributional shifts significantly better than both PCNs and VAEs. In addition, we propose a novel measure of recognizability for individual samples which can be tested against human psychophysical data. Overall, we hope this work will spur interest in iVAEs as a promising new direction for modeling in neuroscience.
University of Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Generative modeling of 3D brain MRIs presents difficulties in achieving high visual fidelity while ensuring sufficient coverage of the data distribution. In this work, we propose to address this challenge with composable, multiscale morphological transformations in a variational autoencoder (VAE) framework. These transformations are applied to a chosen reference brain image to generate MRI volumes, equipping the model with strong anatomical inductive biases. We structure the VAE latent space in a way such that the model covers the data distribution sufficiently well. We show substantial perfor…
Hosei University（法政大学）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
Content-based image retrieval (CBIR) systems are an emerging technology that supports reading and interpreting medical images. Since 3D brain MR images are high dimensional, dimensionality reduction is necessary for CBIR using machine learning techniques. In addition, for a reliable CBIR system, each dimension in the resulting low-dimensional representation must be associated with a neurologically interpretable region. We propose a localized variational autoencoder (Loc-VAE) that provides neuroanatomically interpretable low-dimensional representation from 3D brain MR images for clinical CBIR. …
Multiple Sclerosis (MS) and microvascular leukoencephalopathy are two distinct neurological conditions, the first caused by focal autoimmune inflammation in the central nervous system, the second caused by chronic white matter damage from atherosclerotic microvascular disease. Both conditions lead to signal anomalies on Fluid Attenuated Inversion Recovery (FLAIR) magnetic resonance (MR) images, which can be distinguished by an expert neuroradiologist, but which can look very similar to the untrained eye as well as in the early stage of both diseases. In this paper, we attempt to train a 3-dime…
Brain Structure-Function Fusing Representation Learning using Adversarial Decomposed-VAE for Analyzing MCI
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen University（深圳大学）；Maebashi Institute of Technology（前桥工业大学）
Iterative Encoding-Decoding VAEs Anomaly Detection in NOAA's DART Time Series: A Machine Learning Approach for Enhancing Data Integrity for NASA's GRACE-FO Verification and Validation
University of California at Los Angeles（加利福尼亚大学洛杉矶分校）；NASA Jet Propulsion Laboratory（美国国家航空航天局喷气推进实验室）
TReNDS center（TReNDS中心）；Georgia Institute of Technology（佐治亚理工学院）；Charite University Hospital（夏里特大学医院）；Georgia State University（佐治亚州立大学）
Communication within or between complex systems is commonplace in the natural sciences and fields such as graph neural networks. The brain is a perfect example of such a complex system, where communication between brain regions is constantly being orchestrated. To analyze communication, the brain is often split up into anatomical regions that each perform certain computations. These regions must interact and communicate with each other to perform tasks and support higher-level cognition. On a macroscale, these regions communicate through signal propagation along the cortex and along white matt…
While unsupervised variational autoencoders (VAE) have become a powerful tool in neuroimage analysis, their application to supervised learning is under-explored. We aim to close this gap by proposing a unified probabilistic model for learning the latent space of imaging data and performing supervised regression. Based on recent advances in learning disentangled representations, the novel generative process explicitly models the conditional distribution of latent representations with respect to the regression target variable. Performing a variational inference procedure on this model leads to j…
California Institute of Technology（加州理工学院）；Disney Research（迪士尼研究院）
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computationally efficient. However, standard inference models are restricted to direct mappings from data to approximate posterior estimates. The failure of these models to reach fully optimized approximate posterior estimates results in an amortization gap. We aim toward closing this gap …
Image generative models can learn the distributions of the training data and consequently generate examples by sampling from these distributions. However, when the training dataset is corrupted with outliers, generative models will likely produce examples that are also similar to the outliers. In fact, a small portion of outliers may induce state-of-the-art generative models, such as Vector Quantized-Variational AutoEncoder (VQ-VAE), to learn a significant mode from the outliers. To mitigate this problem, we propose a robust generative model based on VQ-VAE, which we name Robust VQ-VAE (RVQ-VA…
Technical University of Denmark（丹麦技术大学）；University of Copenhagen（哥本哈根大学）
A few-shot generative model should be able to generate data from a novel distribution by only observing a limited set of examples. In few-shot learning the model is trained on data from many sets from distributions sharing some underlying properties such as sets of characters from different alphabets or objects from different categories. We extend current latent variable models for sets to a fully hierarchical approach with an attention-based point to set-level aggregation and call our method SCHA-VAE for Set-Context-Hierarchical-Aggregation Variational Autoencoder. We explore likelihood-based…
Universitee de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；LORIA（洛林计算机科学及其应用实验室）
Recent studies have explored the use of deep generative models of speech spectra based of variational autoencoders (VAEs), combined with unsupervised noise models, to perform speech enhancement. These studies developed iterative algorithms involving either Gibbs sampling or gradient descent at each step, making them computationally expensive. This paper proposes a variational inference method to iteratively estimate the power spectrogram of the clean speech. Our main contribution is the analytical derivation of the variational steps in which the en-coder of the pre-learned VAE can be used to e…
Lehigh University（利哈伊大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）
Normative modeling is an emerging and promising approach to effectively study disorder heterogeneity in individual participants. In this study, we propose a novel normative modeling method by combining conditional variational autoencoder with adversarial learning (ACVAE) to identify brain dysfunction in Alzheimer's Disease (AD). Specifically, we first train a conditional VAE on the healthy control (HC) group to create a normative model conditioned on covariates like age, gender and intracranial volume. Then we incorporate an adversarial training process to construct a discriminative feature sp…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We present an empirical evaluation of fMRI data augmentation via synthesis. For synthesis we use generative mod-els trained on real neuroimaging data to produce novel task-dependent functional brain images. Analyzed generative mod-els include classic approaches such as the Gaussian mixture model (GMM), and modern implicit generative models such as the generative adversarial network (GAN) and the variational auto-encoder (VAE). In particular, the proposed GAN and VAE models utilize 3-dimensional convolutions, which enables modeling of high-dimensional brain image tensors with structured spatial…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We propose a new class of physics-informed neural networks, called physics-informed Variational Autoencoder (PI-VAE), to solve stochastic differential equations (SDEs) or inverse problems involving SDEs. In these problems the governing equations are known but only a limited number of measurements of system parameters are available. PI-VAE consists of a variational autoencoder (VAE), which generates samples of system variables and parameters. This generative model is integrated with the governing equations. In this integration, the derivatives of VAE outputs are readily calculated using automat…
Columbia University（哥伦比亚大学）；UT Austin（德克萨斯大学奥斯汀分校）
The ability to record activities from hundreds of neurons simultaneously in the brain has placed an increasing demand for developing appropriate statistical techniques to analyze such data. Recently, deep generative models have been proposed to fit neural population responses. While these methods are flexible and expressive, the downside is that they can be difficult to interpret and identify. To address this problem, we propose a method that integrates key ingredients from latent models and traditional neural encoding models. Our method, pi-VAE, is inspired by recent progress on identifiable …
Negative Binomial Variational Autoencoders for Overdispersed Latent Modeling
Southeast University（东南大学）；China Medical University Shenyang（中国医科大学沈阳）；University of Chinese Academy of Science（中国科学院大学）；Alibaba Cloud（阿里云）；Center for Applied Statistics and School of Statistics, Renmin University of China（中国人民大学应用统计中心与统计学院）
Statistical Test for Anomaly Detections by Variational Auto-Encoders
Nagoya Institute of Technology（名古屋工业大学）；Nagoya University（名古屋大学）；University of Information Technology（信息技术大学）；Vietnam National University（越南国家大学）；RIKEN（理化学研究所）
Amazon Web Services（亚马逊网络服务公司）；University of Southern California（南加州大学）；Los Angeles, CA, USA（美国加利福尼亚州洛杉矶）
Estimation of uncertainty in deep learning models is of vital importance, especially in medical imaging, where reliance on inference without taking into account uncertainty could lead to misdiagnosis. Recently, the probabilistic Variational AutoEncoder (VAE) has become a popular model for anomaly detection in applications such as lesion detection in medical images. The VAE is a generative graphical model that is used to learn the data distribution from samples and then generate new samples from this distribution. By training on normal samples, the VAE can be used to detect inputs that deviate …
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Variational autoencoders (VAEs), as an important aspect of generative models, have received a lot of research interests and reached many successful applications. However, it is always a challenge to achieve the consistency between the learned latent distribution and the prior latent distribution when optimizing the evidence lower bound (ELBO), and finally leads to an unsatisfactory performance in data generation. In this paper, we propose a latent distribution consistency approach to avoid such substantial inconsistency between the posterior and prior latent distributions in ELBO optimizing. W…
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；Center for Behavioral Brain Sciences（行为脑科学研究中心）；German Center for Neurodegenerative Disease（德国神经退行性疾病研究中心）
Expert interpretation of anatomical images of the human brain is the central part of neuro-radiology. Several machine learning-based techniques have been proposed to assist in the analysis process. However, the ML models typically need to be trained to perform a specific task, e.g., brain tumour segmentation or classification. Not only do the corresponding training data require laborious manual annotations, but a wide variety of abnormalities can be present in a human brain MRI - even more than one simultaneously, which renders representation of all possible anomalies very challenging. Hence, …
University of Florida（佛罗里达大学）；UiT - The Arctic University of Norway（挪威北极圈大学）
We propose causal recurrent variational autoencoder (CR-VAE), a novel generative model that is able to learn a Granger causal graph from a multivariate time series x and incorporates the underlying causal mechanism into its data generation process. Distinct to the classical recurrent VAEs, our CR-VAE uses a multi-head decoder, in which the $p$-th head is responsible for generating the $p$-th dimension of $\mathbf{x}$ (i.e., $\mathbf{x}^p$). By imposing a sparsity-inducing penalty on the weights (of the decoder) and encouraging specific sets of weights to be zero, our CR-VAE learns a sparse adj…
Enhancing the Parameterization of Reservoir Properties for Data Assimilation Using Deep VAE-GAN
Departamento de Engenharia de Minas e de Petróleo, Escola Politécnica, Universidade de São Paulo（圣保罗大学采矿与石油工程系，理工学院）；Department of Earth Science and Engineering, Imperial College London（伦敦帝国理工学院地球科学与工程系）
Automatic MRI brain tumor segmentation is of vital importance for the disease diagnosis, monitoring, and treatment planning. In this paper, we propose a two-stage encoder-decoder based model for brain tumor subregional segmentation. Variational autoencoder regularization is utilized in both stages to prevent the overfitting issue. The second-stage network adopts attention gates and is trained additionally using an expanded dataset formed by the first-stage outputs. On the BraTS 2020 validation dataset, the proposed method achieves the mean Dice score of 0.9041, 0.8350, and 0.7958, and Hausdorf…
Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London（伦敦大学学院惠康/工程与物理科学研究委员会介入与外科科学中心）；Centre For Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；Division of Surgery and Interventional Science, University College London（伦敦大学学院外科与介入科学系）
In this work we propose a method based on geometric deep learning to predict the complete surface of the liver, given a partial point cloud of the organ obtained during the surgical laparoscopic procedure. We introduce a new data augmentation technique that randomly perturbs shapes in their frequency domain to compensate the limited size of our dataset. The core of our method is a variational autoencoder (VAE) that is trained to learn a latent space for complete shapes of the liver. At inference time, the generative part of the model is embedded in an optimisation procedure where the latent re…
Queen’s University（女王大学）；Azad University（阿扎德大学）；University of Leeds（利兹大学）；KU Leuven（荷语鲁汶大学）；Shenzhen University（深圳大学）
In large studies involving multi protocol Magnetic Resonance Imaging (MRI), it can occur to miss one or more sub-modalities for a given patient owing to poor quality (e.g. imaging artifacts), failed acquisitions, or hallway interrupted imaging examinations. In some cases, certain protocols are unavailable due to limited scan time or to retrospectively harmonise the imaging protocols of two independent studies. Missing image modalities pose a challenge to segmentation frameworks as complementary information contributed by the missing scans is then lost. In this paper, we propose a novel model, …
IterMask3D: Unsupervised Anomaly Detection and Segmentation with Test-Time Iterative Mask Refinement in 3D Brain MR
Department of Engineering Science, University of Oxford, UK（工程科学系，牛津大学）；Department of Computer Science, Hong Kong Baptist University（计算机科学系，香港 Baptist 大学）；Nuffield Department of Clinical Neurosciences, University of Oxford（临床神经科学系，牛津大学）；Department of Neuroradiology, Oxford University Hospitals NHS Foundation Trust（放射学系，牛津大学医院 NHS 基础研究信托）
ARC Centre of Excellence for Mathematical & Statistical Frontiers（ARC数学与统计前沿卓越中心）；Queensland University of Technology（昆士兰科技大学）；Beijing Institute of Technology（北京理工大学）；University of Technology Sydney（悉尼科技大学）
Construction of dictionaries using nonnegative matrix factorisation (NMF) has extensive applications in signal processing and machine learning. With the advances in deep learning, training compact and robust dictionaries using deep neural networks, i.e., dictionaries of deep features, has been proposed. In this study, we propose a probabilistic generative model which employs a variational autoencoder (VAE) to perform nonnegative dictionary learning. In contrast to the existing VAE models, we cast the model under a statistical framework with latent variables obeying a Gamma distribution and des…
Graphs are a fundamental abstraction for modeling relational data. However, graphs are discrete and combinatorial in nature, and learning representations suitable for machine learning tasks poses statistical and computational challenges. In this work, we propose Graphite, an algorithmic framework for unsupervised learning of representations over nodes in large graphs using deep latent variable generative models. Our model parameterizes variational autoencoders (VAE) with graph neural networks, and uses a novel iterative graph refinement strategy inspired by low-rank approximations for decoding…
Research on Feature Extraction Data Processing System For MRI of Brain Diseases Based on Computer Deep Learning
Georgia Institute of Technology（佐治亚理工学院）；Arizona State University（亚利桑那州立大学）；University of California, Davis（加州大学戴维斯分校）；Columbia University（哥伦比亚大学）；Canoakbit Alliance Inc（Canoakbit Alliance 公司）；North Carolina State University（北卡罗来纳州立大学）
In this paper we introduce a recurrent neural network (RNN) based variational autoencoder (VAE) model with a new constrained loss function that can generate more meaningful electroencephalography (EEG) features from raw EEG features to improve the performance of EEG based speech recognition systems. We demonstrate that both continuous and isolated speech recognition systems trained and tested using EEG features generated from raw EEG features using our VAE model results in improved performance and we demonstrate our results for a limited English vocabulary consisting of 30 unique sentences for…
Multiparameter Uncertainty Mapping in Quantitative Molecular MRI using a Physics-Structured Variational Autoencoder (PS-VAE)
School of Biomedical Engineering, Tel-Aviv University（生物医学工程学院，特拉维夫大学）；School of Biochemistry, Neurobiology, and Biophysics, Tel-Aviv University（生物化学、神经生物学与生物物理学学院，特拉维夫大学）；Institute of Neuroradiology, University Hospital Erlangen（神经放射学研究所，埃朗根大学医院）；Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander Universität Erlangen-Nürnberg (FAU)（生物医学工程人工智能系，弗赖堡-艾尔朗根-纽伦堡大学（FAU））；Sagol School of Neuroscience, Tel-Aviv University（萨戈尔神经科学学院，特拉维夫大学）
Samsung AI Center（三星人工智能中心）；Rutgers University（罗格斯大学）
Inference networks of traditional Variational Autoencoders (VAEs) are typically amortized, resulting in relatively inaccurate posterior approximation compared to instance-wise variational optimization. Recent semi-amortized approaches were proposed to address this drawback; however, their iterative gradient update procedures can be computationally demanding. To address these issues, in this paper we introduce an accurate amortized inference algorithm. We propose a novel recursive mixture estimation algorithm for VAEs that iteratively augments the current mixture with new components so as to ma…
City, University of London（伦敦大学城市学院）；Imperial College（帝国理工学院）
We propose a novel unsupervised out-of-distribution detection method for medical images based on implicit fields image representations. In our approach, an auto-decoder feed-forward neural network learns the distribution of healthy images in the form of a mapping between spatial coordinates and probabilities over a proxy for tissue types. At inference time, the learnt distribution is used to retrieve, from a given test image, a restoration, i.e. an image maximally consistent with the input one but belonging to the healthy distribution. Anomalies are localized using the voxel-wise probability p…
Anomalies are by definition rare, thus labeled examples are very limited or nonexistent, and likely do not cover unforeseen scenarios. Unsupervised learning methods that don't necessarily encounter anomalies in training would be immensely useful. Generative vision models can be useful in this regard but do not sufficiently represent normal and abnormal data distributions. To this end, we propose constant curvature manifolds for embedding data distributions in unsupervised visual anomaly detection. Through theoretical and empirical explorations of manifold shapes, we develop a novel hyperspheri…
Improving Normative Modeling for Multi-modal Neuroimaging Data using mixture-of-product-of-experts variational autoencoders
Department of Computer Science and Engineering, Washington University in St. Louis, USA（计算机科学与工程系，华盛顿大学圣路易斯分校）；Institute for Informatics, Data Science and Biostatistics, Washington University in St.Louis, USA（信息学、数据科学与生物统计研究所，华盛顿大学圣路易斯分校）；Department of Radiology, Washington University in St.Louis, USA（放射学系，华盛顿大学圣路易斯分校）
Matrix factorization (MF) has been widely applied to collaborative filtering in recommendation systems. Its Bayesian variants can derive posterior distributions of user and item embeddings, and are more robust to sparse ratings. However, the Bayesian methods are restricted by their update rules for the posterior parameters due to the conjugacy of the priors and the likelihood. Variational autoencoders (VAE) can address this issue by capturing complex mappings between the posterior parameters and the data. However, current research on VAEs for collaborative filtering only considers the mappings…
Isolating Nonlinear Independent Sources in fMRI with $β$-TCVAE Models
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science（转化神经影像与数据科学跨机构中心）；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Valencia（瓦伦西亚大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory（三机构神经影像与数据科学转化研究中心（TReNDS），佐治亚州立大学、佐治亚理工学院、埃默里大学）；Faculty of Electrical Engineering, Mathematics & Computer Science, TU Delft（代尔夫特理工大学电气工程、数学与计算机科学学院）；Lambert High School（兰伯特高中）
Neuroimaging studies often involve the collection of multiple data modalities. These modalities contain both shared and mutually exclusive information about the brain. This work aims at finding a scalable and interpretable method to fuse the information of multiple neuroimaging modalities using a variational autoencoder (VAE). To provide an initial assessment, this work evaluates the representations that are learned using a schizophrenia classification task. A support vector machine trained on the representations achieves an area under the curve for the classifier's receiver operating characte…
