A Hierarchical Graphical Model for Big Inverse Covariance Estimation with an Application to fMRI
Xi Luo
备注：
An R package of the proposed method will be publicly available on CRAN. This paper has been presented orally at Yale University on Feburary 18, 2014, and at the Eastern North American Region Meeting of the International Biometric Society on March 18, 2014
Brain networks has attracted the interests of many neuroscientists. From functional MRI (fMRI) data, statistical tools have been developed to recover brain networks. However, the dimensionality of whole-brain fMRI, usually in hundreds of thousands, challenges the applicability of these methods. We develop a hierarchical graphical model (HGM) to remediate this difficulty. This model introduces a hidden layer of networks based on sparse Gaussian graphical models, and the observed data are sampled from individual network nodes. In fMRI, the network layer models the underlying signals of different brain functional units, and how these units directly interact with each other. The introduction of this hierarchical structure not only provides a formal and interpretable approach, but also enables efficient computation for inferring big networks with hundreds of thousands of nodes. Based on the conditional convexity of our formulation, we develop an alternating update algorithm to compute the HGM model parameters simultaneously. The effectiveness of this approach is demonstrated on simulated data and a real dataset from a stop/go fMRI experiment.
Neuroimaging is the growing area of neuroscience devoted to produce data with the goal of capturing processes and dynamics of the human brain. We consider the problem of inferring the brain connectivity network from time dependent functional magnetic resonance imaging (fMRI) scans. To this aim we propose the symmetric graphical lasso, a penalized likelihood method with a fused type penalty function that takes into explicit account the natural symmetrical structure of the brain. Symmetric graphical lasso allows one to learn simultaneously both the network structure and a set of symmetries acros…
We develop a new estimator of the inverse covariance matrix for high-dimensional multivariate normal data using the horseshoe prior. The proposed graphical horseshoe estimator has attractive properties compared to other popular estimators, such as the graphical lasso and graphical Smoothly Clipped Absolute Deviation (SCAD). The most prominent benefit is that when the true inverse covariance matrix is sparse, the graphical horseshoe provides estimates with small information divergence from the true sampling distribution. The posterior mean under the graphical horseshoe prior can also be almost …
We introduce the Poisson Log-Normal Graphical Model for count data, and present a normality transformation for data arising from this distribution. The model and transformation are feasible for high-throughput microRNA (miRNA) sequencing data and directly account for known overdispersion relationships present in this data set. The model allows for network dependencies to be modeled, and we provide an algorithm which utilizes a one-step EM based result in order to allow for a provable increase in performance in determining the network structure. The model is shown to provide an increase in perf…
Large-scale precision matrix estimation is of fundamental importance yet challenging in many contemporary applications for recovering Gaussian graphical models. In this paper, we suggest a new approach of innovated scalable efficient estimation (ISEE) for estimating large precision matrix. Motivated by the innovated transformation, we convert the original problem into that of large covariance matrix estimation. The suggested method combines the strengths of recent advances in high-dimensional sparse modeling and large covariance matrix estimation. Compared to existing approaches, our method is…
Anomalies and outliers are common in real-world data, and they can arise from many sources, such as sensor faults. Accordingly, anomaly detection is important both for analyzing the anomalies themselves and for cleaning the data for further analysis of its ambient structure. Nonetheless, a precise definition of anomalies is important for automated detection and herein we approach such problems from the perspective of detecting sparse latent effects embedded in large collections of noisy data. Standard Graphical Lasso-based techniques can identify the conditional dependency structure of a colle…
In this paper, we introduce a new directed graphical model from Gaussian data: the Gaussian graphical interaction model (GGIM). The development of this model comes from considering stationary Gaussian processes on graphs, and leveraging the equations between the resulting steady-state covariance matrix and the Laplacian matrix representing the interaction graph. Through the presentation of conceptually straightforward theory, we develop the new model and provide interpretations of the edges in the graphical model in terms of statistical measures. We show that when restricted to undirected grap…
KU Leuven（鲁汶大学）；Leuven Statistics Research Center (LStat)（鲁汶统计研究中心）；Université Paris Descartes（巴黎笛卡尔大学）；CNRS（法国国家科学研究中心）；Université Paris-Sud（巴黎南大学）；INRA（法国国家农业研究院）
Gaussian graphical models are widely utilized to infer and visualize networks of dependencies between continuous variables. However, inferring the graph is difficult when the sample size is small compared to the number of variables. To reduce the number of parameters to estimate in the model, we propose a non-asymptotic model selection procedure supported by strong theoretical guarantees based on an oracle inequality and a minimax lower bound. The covariance matrix of the model is approximated by a block-diagonal matrix. The structure of this matrix is detected by thresholding the sample covar…
The interactions between microbial taxa in microbiome data has been under great research interest in the science community. In particular, several methods such as SPIEC-EASI, gCoda, and CD-trace have been proposed to model the conditional dependency between microbial taxa, in order to eliminate the detection of spurious correlations. However, all those methods are built upon the central log-ratio (CLR) transformation, which results in a degenerate covariance matrix and thus an undefined inverse covariance matrix as the estimation of the underlying network. Jiang et al. (2021) and Tian et al. (…
Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）；YerevaNN Research Lab（YerevaNN研究实验室）
Estimating the covariance structure of multivariate time series is a fundamental problem with a wide-range of real-world applications -- from financial modeling to fMRI analysis. Despite significant recent advances, current state-of-the-art methods are still severely limited in terms of scalability, and do not work well in high-dimensional undersampled regimes. In this work we propose a novel method called Temporal Correlation Explanation, or T-CorEx, that (a) has linear time and memory complexity with respect to the number of variables, and can scale to very large temporal datasets that are n…
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Building on a recent framework for distributionally robust optimization, we consider estimation of the inverse covariance matrix for multivariate data. We provide a novel notion of a Wasserstein ambiguity set specifically tailored to this estimation problem, leading to a tractable class of regularized estimators. Special cases include penalized likelihood estimators for Gaussian data, specifically the graphical lasso estimator. As a consequence of this formulation, the radius of the Wasserstein ambiguity set is directly related to the regularization parameter in the estimation problem. Using t…
In this work, we present an additive model for space-time data that splits the data into a temporally correlated component and a spatially correlated component. We model the spatially correlated portion using a time-varying Gaussian graphical model. Under assumptions on the smoothness of changes in covariance matrices, we derive strong single sample convergence results, confirming our ability to estimate meaningful graphical structures as they evolve over time. We apply our methodology to the discovery of time-varying spatial structures in human brain fMRI signals.
High dimensional covariance estimation and graphical models is a contemporary topic in statistics and machine learning having widespread applications. An important line of research in this regard is to shrink the extreme spectrum of the covariance matrix estimators. A separate line of research in the literature has considered sparse inverse covariance estimation which in turn gives rise to graphical models. In practice, however, a sparse covariance or inverse covariance matrix which is simultaneously well-conditioned and at the same time computationally tractable is desired. There has been lit…
The precision matrix that encodes conditional linear dependency relations among a set of variables forms an important object of interest in multivariate analysis. Sparse estimation procedures for precision matrices such as the graphical lasso (Glasso) gained popularity as they facilitate interpretability, thereby separating pairs of variables that are conditionally dependent from those that are independent (given all other variables). Glasso lacks, however, robustness to outliers. To overcome this problem, one typically applies a robust plug-in procedure where the Glasso is computed from a rob…
Graphical Lasso (GL) is a popular method for learning the structure of an undirected graphical model, which is based on an $l_1$ regularization technique. The objective of this paper is to compare the computationally-heavy GL technique with a numerically-cheap heuristic method that is based on simply thresholding the sample covariance matrix. To this end, two notions of sign-consistent and inverse-consistent matrices are developed, and then it is shown that the thresholding and GL methods are equivalent if: (i) the thresholded sample covariance matrix is both sign-consistent and inverse-consis…
Covariance estimation is ubiquitous in functional data analysis. Yet, the case of functional observations over multidimensional domains introduces computational and statistical challenges, rendering the standard methods effectively inapplicable. To address this problem, we introduce "Covariance Networks" (CovNet) as a modeling and estimation tool. The CovNet model is "universal" - it can be used to approximate any covariance up to desired precision. Moreover, the model can be fitted efficiently to the data and its neural network architecture allows us to employ modern computational tools in th…
An Evaluation of Sparse Inverse Covariance Models for Group Functional Connectivity in Molecular Imaging
通过神经成像技术测量大脑区域之间的功能关系，可以深入了解大脑如何在宏观尺度上共享信息。许多功能性连接方法已被开发用于动态成像模式，如功能性MRI（fMRI），而较少的工作集中在静态分子成像技术，如FDG-PET和Tc-99 m HMPAO SPECT跨人群的模型。在这项工作中，我们提供了一个定量评估如何以及三个功能connectivity模型的基础上稀疏逆协方差估计可以准确地恢复黄金标准连接模式跨多个队列和数据集大小。我们比较了学习正则化逆协方差阵的准确性，这些学习独立于使用两个不同的基于组的正则化模型。通过使用大量的SPECT扫描，我们能够对模型在恢复金标准功能连接方面的准确性进行定量评估。
This paper introduces the Sylvester graphical lasso (SyGlasso) that captures multiway dependencies present in tensor-valued data. The model is based on the Sylvester equation that defines a generative model. The proposed model complements the tensor graphical lasso (Greenewald et al., 2019) that imposes a Kronecker sum model for the inverse covariance matrix by providing an alternative Kronecker sum model that is generative and interpretable. A nodewise regression approach is adopted for estimating the conditional independence relationships among variables. The statistical convergence of the m…
The sparse inverse covariance estimation problem is commonly solved using an $\ell_{1}$-regularized Gaussian maximum likelihood estimator known as "graphical lasso", but its computational cost becomes prohibitive for large data sets. A recent line of results showed--under mild assumptions--that the graphical lasso estimator can be retrieved by soft-thresholding the sample covariance matrix and solving a maximum determinant matrix completion (MDMC) problem. This paper proves an extension of this result, and describes a Newton-CG algorithm to efficiently solve the MDMC problem. Assuming that the…
We consider estimation of undirected Gaussian graphical models and inverse covariances in high-dimensional scenarios by penalizing the corresponding precision matrix. While single $L_1$ (Graphical Lasso) and $L_2$ (Graphical Ridge) penalties for the precision matrix have already been studied, we propose the combination of both, yielding an Elastic Net type penalty. We enable additional flexibility by allowing to include diagonal target matrices for the precision matrix. We generalize existing algorithms for the Graphical Lasso and provide corresponding software with an efficient implementation…
We propose the use of a robust covariance estimator based on multivariate Winsorization in the context of the Tarr-Muller-Weber framework for sparse estimation of the precision matrix of a Gaussian graphical model. Likewise Croux-Ollerer's precision matrix estimator, our proposed estimator attains the maximum finite sample breakdown point of 0.5 under cellwise contamination. We conduct an extensive Monte Carlo simulation study to assess the performance of ours and the currently existing proposals. We find that ours has a competitive behavior, regarding the the estimation of the precision matri…
Gaussian graphical models (GGMs) are probabilistic tools of choice for analyzing conditional dependencies between variables in complex systems. Finding changepoints in the structural evolution of a GGM is therefore essential to detecting anomalies in the underlying system modeled by the GGM. In order to detect structural anomalies in a GGM, we consider the problem of estimating changes in the precision matrix of the corresponding Gaussian distribution. We take a two-step approach to solving this problem:- (i) estimating a background precision matrix using system observations from the past with…
A precision matrix is the inverse of a covariance matrix. In this paper, we study the problem of estimating the precision matrix with a known graphical structure under high-dimensional settings. We propose a simple estimator of the precision matrix based on the connection between the known graphical structure and the precision matrix. We obtain the rates of convergence of the proposed estimators and derive the asymptotic normality of the proposed estimator in the high-dimensional setting when the data dimension grows with the sample size. Numerical simulations are conducted to demonstrate the …
Graphical models have been used extensively for modeling brain connectivity networks. However, unmeasured confounders and correlations among measurements are often overlooked during model fitting, which may lead to spurious scientific discoveries. Motivated by functional magnetic resonance imaging (fMRI) studies, we propose a novel method for constructing brain connectivity networks with correlated replicates and latent effects. In a typical fMRI study, each participant is scanned and fMRI measurements are collected across a period of time. In many cases, subjects may have different states of …
This paper considers a novel problem, bi-level graphical modeling, in which multiple individual graphical models can be considered as variants of a common group-level graphical model and inference of both the group- and individual-level graphical models are of interest. Such problem arises from many applications including multi-subject neuroimaging and genomics data analysis. We propose a novel and efficient statistical method, the random covariance model, to learn the group- and individual-level graphical models simultaneously. The proposed method can be nicely interpreted as a random covaria…
Florida State University（佛罗里达州立大学）；University of South Florida（南佛罗里达大学）
Complex analyses involving multiple, dependent random quantities often lead to graphical models - a set of nodes denoting variables of interest, and corresponding edges denoting statistical interactions between nodes. To develop statistical analyses for graphical data, especially towards generative modeling, one needs mathematical representations and metrics for matching and comparing graphs, and subsequent tools, such as geodesics, means, and covariances. This paper utilizes a quotient structure to develop efficient algorithms for computing these quantities, leading to useful statistical tool…
In applications of graphical models, we typically have more information than just the samples themselves. A prime example is the estimation of brain connectivity networks based on fMRI data, where in addition to the samples themselves, the spatial positions of the measurements are readily available. With particular regard for this application, we are thus interested in ways to incorporate additional knowledge most effectively into graph estimation. Our approach to this is to make neighborhood selection receptive to additional knowledge by strengthening the role of the tuning parameters. We dem…
Multichannel electroencephalograms (EEGs) have been widely used to study cortical connectivity during acquisition of motor skills. In this paper, we introduce copula Gaussian graphical models on spectral domain to characterize dependence in oscillatory activity between channels. To obtain a simple and robust representation of brain connectivity that can explain the most variation in the observed signals, we propose a framework based on maximizing penalized likelihood with Lasso regularization to search for the sparse precision matrix. To address the optimization problem, graphical Lasso, Ledoi…
Gaussian graphical models are used for determining conditional relationships between variables. This is accomplished by identifying off-diagonal elements in the inverse-covariance matrix that are non-zero. When the ratio of variables (p) to observations (n) approaches one, the maximum likelihood estimator of the covariance matrix becomes unstable and requires shrinkage estimation. Whereas several classical (frequentist) methods have been introduced to address this issue, fully Bayesian methods remain relatively uncommon in practice and methodological literatures. Here we introduce a Bayesian m…
