We propose two model-free, permutation-based tests of independence between a pair of random variables. The tests can be applied to samples from any bivariate distribution: continuous, discrete or mixture of those, with light tails or heavy tails, \ldots The tests take advantage of the recent development of the global envelope tests in the context of spatial statistics. Apart from the broad applicability of the tests, their main benefit lies in the graphical interpretation of the test outcome: in case of rejection of the null hypothesis of independence, the combinations of quantiles in the two marginals are indicated for which the deviation from independence is significant. This information can be used to gain more insight into the properties of the observed data and as a guidance for proposing more complicated models and hypotheses. We assess the performance of the proposed tests in a simulation study and compare them to several well-established tests of independence. Furthermore, we illustrate the use of the tests and the interpretation of the test outcome in two real datasets consisting of meteorological reports (daily mean temperature and total daily precipitation, having an atomic component at 0 millimeters) and road accidents reports (type of road and the weather conditions, both variables having categorical distribution).
Graphical interaction models have become an important tool for analysing multivariate time series. In these models, the interrelationships among the components of a time series are described by undirected graphs in which the vertices depict the components while the edges indictate possible dependencies between the components. Current methods for the identification of the graphical structure are based on nonparametric spectral stimation, which prevents application of common model selection strategies. In this paper, we present a parametric approach for graphical interaction modelling of multiva…
Model selection and learning the structure of graphical models from the data sample constitutes an important field of probabilistic graphical model research, as in most of the situations the structure is unknown and has to be learnt from the given dataset. In this paper, we present a new forward model selection algorithm for graphical log-linear models. We use mutual conditional independence check to reduce the search space which also takes care of the evaluation of the joint effects and chances of missing important interactions are eliminated. We illustrate our algorithm with a real dataset e…
Graphical models are a key class of probabilistic models for studying the conditional independence structure of a set of random variables. Circular variables are special variables, characterized by periodicity, arising in several contexts and fields. However, models for studying the dependence/independence structure of circular variables are under-explored. This paper analyses three multivariate circular distributions, the von Mises, the Wrapped Normal and the Inverse Stereographic distributions, focusing on their properties concerning conditional independence. For each one of these distributi…
We extend the log-mean linear parameterization introduced by Roverato et al. (2013) for binary data to discrete variables with arbitrary number of levels, and show that also in this case it can be used to parameterize bi-directed graph models. Furthermore, we show that the log-mean linear parameterization allows one to simultaneously represent marginal independencies among variables and marginal independencies that only appear when certain levels are collapsed into a single one. We illustrate the application of this property by means of an example based on genetic association studies involving…
We formulate and analyze a graphical model selection method for inferring the conditional independence graph of a high-dimensional nonstationary Gaussian random process (time series) from a finite-length observation. The observed process samples are assumed uncorrelated over time and having a time-varying marginal distribution. The selection method is based on testing conditional variances obtained for small subsets of process components. This allows to cope with the high-dimensional regime, where the sample size can be (drastically) smaller than the process dimension. We characterize the requ…
For multivariate spatial Gaussian process (GP) models, customary specifications of cross-covariance functions do not exploit relational inter-variable graphs to ensure process-level conditional independence among the variables. This is undesirable, especially for highly multivariate settings, where popular cross-covariance functions such as the multivariate Matérn suffer from a "curse of dimensionality" as the number of parameters and floating point operations scale up in quadratic and cubic order, respectively, in the number of variables. We propose a class of multivariate "Graphical Gaussian…
We consider the problem of inferring the conditional independence graph (CIG) of a multivariate stationary dicrete-time Gaussian random process based on a finite length observation. Using information-theoretic methods, we derive a lower bound on the error probability of any learning scheme for the underlying process CIG. This bound, in turn, yields a minimum required sample-size which is necessary for any algorithm regardless of its computational complexity, to reliably select the true underlying CIG. Furthermore, by analysis of a simple selection scheme, we show that the information-theoretic…
Graphical models with bi-directed edges (<->) represent marginal independence: the absence of an edge between two vertices indicates that the corresponding variables are marginally independent. In this paper, we consider maximum likelihood estimation in the case of continuous variables with a Gaussian joint distribution, sometimes termed a covariance graph model. We present a new fitting algorithm which exploits standard regression techniques and establish its convergence properties. Moreover, we contrast our procedure to existing estimation methods.
Bayesian methods for graphical log-linear marginal models have not been developed in the same extent as traditional frequentist approaches. In this work, we introduce a novel Bayesian approach for quantitative learning for such models. These models belong to curved exponential families that are difficult to handle from a Bayesian perspective. Furthermore, the likelihood cannot be analytically expressed as a function of the marginal log-linear interactions, but only in terms of cell counts or probabilities. Posterior distributions cannot be directly obtained, and MCMC methods are needed. Finall…
Statisticians and quantitative neuroscientists have actively promoted the use of independence relationships for investigating brain networks, genomic networks, and other measurement technologies. Estimation of these graphs depends on two steps. First is a feature extraction by summarizing measurements within a parcellation, regional or set definition to create nodes. Secondly, these summaries are then used to create a graph representing relationships of interest. In this manuscript we study the impact of dimension reduction on graphs that describe different notions of relations among a set of …
Testing (conditional) independence of multivariate random variables is a task central to statistical inference and modelling in general - though unfortunately one for which to date there does not exist a practicable workflow. State-of-art workflows suffer from the need for heuristic or subjective manual choices, high computational complexity, or strong parametric assumptions. We address these problems by establishing a theoretical link between multivariate/conditional independence testing, and model comparison in the multivariate predictive modelling aka supervised learning task. This link all…
We introduce a nonparametric graphical model for discrete node variables based on additive conditional independence. Additive conditional independence is a three way statistical relation that shares similar properties with conditional independence by satisfying the semi-graphoid axioms. Based on this relation we build an additive graphical model for discrete variables that does not suffer from the restriction of a parametric model such as the Ising model. We develop an estimator of the new graphical model via the penalized estimation of the discrete version of the additive precision operator a…
Despite major methodological developments, Bayesian inference for Gaussian graphical models remains challenging in high dimension due to the tremendous size of the model space. This article proposes a method to infer the marginal and conditional independence structures between variables by multiple testing of hypotheses. Specifically, we introduce closed-form Bayes factors under the Gaussian conjugate model to evaluate the null hypotheses of marginal and conditional independence between variables. Their computation for all pairs of variables is shown to be extremely efficient, thereby allowing…
Graphical models express conditional independence relationships among variables. Although methods for vector-valued data are well established, functional data graphical models remain underdeveloped. We introduce a notion of conditional independence between random functions, and construct a framework for Bayesian inference of undirected, decomposable graphs in the multivariate functional data context. This framework is based on extending Markov distributions and hyper Markov laws from random variables to random processes, providing a principled alternative to naive application of multivariate m…
This paper introduces a novel class of models for binary data, which we call log-mean linear models. The characterizing feature of these models is that they are specified by linear constraints on the log-mean linear parameter, defined as a log-linear expansion of the mean parameter of the multivariate Bernoulli distribution. We show that marginal independence relationships between variables can be specified by setting certain log-mean linear interactions to zero and, more specifically, that graphical models of marginal independence are log-mean linear models. Our approach overcomes some drawba…
Graphical models have long been studied in statistics as a tool for inferring conditional independence relationships among a large set of random variables. The most existing works in graphical modeling focus on the cases that the data are Gaussian or mixed and the variables are linearly dependent. In this paper, we propose a double regression method for learning graphical models under the high-dimensional nonlinear and non-Gaussian setting, and prove that the proposed method is consistent under mild conditions. The proposed method works by performing a series of nonparametric conditional indep…
This paper deals with the Bayesian analysis of graphical models of marginal independence for three way contingency tables. We use a marginal log-linear parametrization, under which the model is defined through suitable zero-constraints on the interaction parameters calculated within marginal distributions. We undertake a comprehensive Bayesian analysis of these models, involving suitable choices of prior distributions, estimation, model determination, as well as the allied computational issues. The methodology is illustrated with reference to two real data sets.
Graphical modeling explores dependences among a collection of variables by inferring a graph that encodes pairwise conditional independences. For jointly Gaussian variables, this translates into detecting the support of the precision matrix. Many modern applications feature high-dimensional and contaminated data that complicate this task. In particular, traditional robust methods that down-weight entire observation vectors are often inappropriate as high-dimensional data may feature partial contamination in many observations. We tackle this problem by giving a robust method for sparse precisio…
A conditional independence graph is a concise representation of pairwise conditional independence among many variables. Graphical Random Forests (GRaFo) are a novel method for estimating pairwise conditional independence relationships among mixed-type, i.e. continuous and discrete, variables. The number of edges is a tuning parameter in any graphical model estimator and there is no obvious number that constitutes a good choice. Stability Selection helps choosing this parameter with respect to a bound on the expected number of false positives (error control). The performance of GRaFo is evaluat…
Graphical network inference is used in many fields such as genomics or ecology to infer the conditional independence structure between variables, from measurements of gene expression or species abundances for instance. In many practical cases, not all variables involved in the network have been observed, and the samples are actually drawn from a distribution where some variables have been marginalized out. This challenges the sparsity assumption commonly made in graphical model inference, since marginalization yields locally dense structures, even when the original network is sparse. We presen…
High-dimensional Functional Graphical Model Structure Learning via Neighborhood Selection Approach
Booth School of Business, The University of Chicago（芝加哥大学布斯商学院）；Department of Statistics and Data Science, Cornell University（康奈尔大学统计与数据科学系）；Department of Data Sciences and Operations, Marshall School of Business, University of Southern California（南加州大学马歇尔商学院数据科学与运营系）
Measuring conditional dependence is an important topic in statistics with broad applications including graphical models. Under a factor model setting, a new conditional dependence measure based on projection is proposed. The corresponding conditional independence test is developed with the asymptotic null distribution unveiled where the number of factors could be high-dimensional. It is also shown that the new test has control over the asymptotic significance level and can be calculated efficiently. A generic method for building dependency graphs without Gaussian assumption using the new test …
Causal inference methods based on conditional independence construct Markov equivalent graphs, and cannot be applied to bivariate cases. The approaches based on independence of cause and mechanism state, on the contrary, that causal discovery can be inferred for two observations. In our contribution, we challenge to reconcile these two research directions. We study the role of latent variables such as latent instrumental variables and hidden common causes in the causal graphical structures. We show that the methods based on the independence of cause and mechanism, indirectly contain traces of …
We develop an asymptotic theory for extremes in decomposable graphical models by presenting results applicable to a range of extremal dependence types. Specifically, we investigate the weak limit of the distribution of suitably normalised random vectors, conditioning on an extreme component, where the conditional independence relationships of the random vector are described by a chordal graph. Under mild assumptions, the random vector corresponding to the distribution in the weak limit, termed the tail graphical model, inherits the graphical structure of the original chordal graph. Our theory …
Testing independence and conditional independence in high dimensions via coordinatewise Gaussianization
Southwestern University of Finance and Economics（西南财经大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；The London School of Economics and Political Science（伦敦政治经济学院）
From Conditional to Unconditional Independence: Testing Conditional Independence via Transport Maps
Institute of Statistics and Big Data, Renmin University of China（中国人民大学统计与大数据研究院）；Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University（香港理工大学数据科学与人工智能学院）；Department of Applied Mathematics, The Hong Kong Polytechnic University（香港理工大学应用数学系）
