A new class of colored Gaussian graphical models with explicit normalizing constants
Warsaw University of Technology（华沙理工大学）；Univ Angers, CNRS, LAREMA, SFR MATHSTIC（昂热大学、法国国家科学研究中心、LAREMA、SFR MATHSTIC）；Department of Mathematics, Osaka Metropolitan University（大阪公立大学数学系）
Adam Chojecki, Piotr Graczyk, Hideyuki Ishi, Bartosz Kołodziejek
备注：
70 pages. Revised exposition, proofs, and examples
We study Bayesian model selection in colored Gaussian graphical models (CGGMs), which combine sparsity of conditional independencies with symmetry constraints encoded by vertex- and edge-colored graphs. A key computational bottleneck in Bayesian inference for CGGMs is the evaluation of the Diaconis-Ylvisaker normalizing constants, given by gamma-type integrals over cones of precision matrices with prescribed zeros and equality constraints.
We introduce a new subclass of RCON models for which these normalizing constants admit closed-form expressions. On the algebraic side, we identify conditions on the space of colored precision matrices that guarantee tractability of the associated integrals, leading to the notions of Block-Cholesky spaces (BC-spaces) and Diagonally Commutative Block-Cholesky spaces (DCBC-spaces). On the combinatorial side, we characterize the colored graphs inducing such spaces via a color perfect elimination ordering and a 2-path regularity condition, and define the resulting Color Elimination-Regular (CER) graphs and their symmetric variants. Our framework extends the theory of uncolored decomposable graphs to the colored setting, and the CER class contains all RCOP models associated with decomposable graphs. For models with a single vertex color, our framework reveals a close connection between DCBC-spaces and Bose-Mesner algebras.
For models defined on BC-spaces, we derive finite product formulas for the normalizing constants in terms of gamma functions, matrix determinants, and model-specific structure constants. For RCON models whose matrix spaces are DCBC-spaces, we provide efficient methods for computing all structure constants and determinant factors required to evaluate these formulas. Our results substantially broaden the range of CGGMs amenable to Bayesian structure learning in high-dimensional applications.
Gaussian graphical models have received considerable attention during the past four decades from the statistical and machine learning communities. In Bayesian treatments of this model, the G-Wishart distribution serves as the conjugate prior for inverse covariance matrices satisfying graphical constraints. While it is straightforward to posit the unnormalized densities, the normalizing constants of these distributions have been known only for graphs that are chordal, or decomposable. Up until now, it was unknown whether the normalizing constant for a general graph could be represented explicit…
Normalizing flows model complex probability distributions by combining a base distribution with a series of bijective neural networks. State-of-the-art architectures rely on coupling and autoregressive transformations to lift up invertible functions from scalars to vectors. In this work, we revisit these transformations as probabilistic graphical models, showing they reduce to Bayesian networks with a pre-defined topology and a learnable density at each node. From this new perspective, we propose the graphical normalizing flow, a new invertible transformation with either a prescribed or a lear…
We consider a class of colored graphical Gaussian models obtained by placing symmetry constraints on the precision matrix in a Bayesian framework. The prior distribution on the precision matrix is the colored $G$-Wishart prior which is the Diaconis-Ylvisaker conjugate prior. In this paper, we develop a computationally efficient model search algorithm which combines linear regression with a double reversible jump Markov chain Monte Carlo (MCMC) method. The latter is to estimate the Bayes factors expressed as the ratio of posterior probabilities of two competing models. We also establish the asy…
Gaussian graphical models are relevant tools to learn conditional independence structure between variables. In this class of models, Bayesian structure learning is often done by search algorithms over the graph space. The conjugate prior for the precision matrix satisfying graphical constraints is the well-known G-Wishart. With this prior, the transition probabilities in the search algorithms necessitate evaluating the ratios of the prior normalizing constants of G-Wishart. In moderate to high-dimensions, this ratio is often approximated using sampling-based methods as computationally expensiv…
We consider multivariate centered Gaussian models for the random vector $(Z^1,\ldots, Z^p)$, whose conditional structure is described by a homogeneous graph and which is invariant under the action of a permutation subgroup. The following paper concerns with model selection within colored graphical Gaussian models, when the underlying conditional dependency graph is known. We derive an analytic expression of the normalizing constant of the Diaconis-Ylvisaker conjugate prior for the precision parameter and perform Bayesian model selection in the class of graphical Gaussian models invariant by th…
Normalizing flows have emerged as an important family of deep neural networks for modelling complex probability distributions. In this note, we revisit their coupling and autoregressive transformation layers as probabilistic graphical models and show that they reduce to Bayesian networks with a pre-defined topology and a learnable density at each node. From this new perspective, we provide three results. First, we show that stacking multiple transformations in a normalizing flow relaxes independence assumptions and entangles the model distribution. Second, we show that a fundamental leap of ca…
In Gaussian graphical models, the zero entries in the precision matrix determine the dependence structure, so estimating that sparse precision matrix and, thereby, learning this underlying structure, is an important and challenging problem. We propose an empirical version of the $G$-Wishart prior for sparse precision matrices, where the prior mode is informed by the data in a suitable way. Paired with a prior on the graph structure, a marginal posterior distribution for the same is obtained that takes the form of a ratio of two $G$-Wishart normalizing constants. We show that this ratio can be …
We propose Bayesian methods for Gaussian graphical models that lead to sparse and adaptively shrunk estimators of the precision (inverse covariance) matrix. Our methods are based on lasso-type regularization priors leading to parsimonious parameterization of the precision matrix, which is essential in several applications involving learning relationships among the variables. In this context, we introduce a novel type of selection prior that develops a sparse structure on the precision matrix by making most of the elements exactly zero, in addition to ensuring positive definiteness -- thus cond…
The G-Wishart distribution is the conjugate prior for precision matrices that encode the conditional independencies of a Gaussian graphical model. While the distribution has received considerable attention, posterior inference has proven computationally challenging, in part due to the lack of a direct sampler. In this note, we rectify this situation. The existence of a direct sampler offers a host of new possibilities for the use of G-Wishart variates. We discuss one such development by outlining a new transdimensional model search algorithm--which we term double reversible jump--that leverage…
The Gaussian Graphical Model (GGM) is a popular tool for incorporating sparsity into joint multivariate distributions. The G-Wishart distribution, a conjugate prior for precision matrices satisfying general GGM constraints, has now been in existence for over a decade. However, due to the lack of a direct sampler, its use has been limited in hierarchical Bayesian contexts, relegating mixing over the class of GGMs mostly to situations involving standard Gaussian likelihoods. Recent work, however, has developed methods that couple model and parameter moves, first through reversible jump methods a…
Colored Markov Random Fields for Probabilistic Topological Modeling
Statistical Sciences Dept., Sapienza University of Rome, Italy（统计科学系，罗马萨皮恩扎大学）；DIET Dept., Sapienza University of Rome, Italy（DIET系，罗马萨皮恩扎大学）；DIAG Dept., Sapienza University of Rome, Italy（DIAG系，罗马萨皮恩扎大学）
Pairwise Markov Random Fields (MRFs) or undirected graphical models are parsimonious representations of joint probability distributions. Variables correspond to nodes of a graph, with edges between nodes corresponding to conditional dependencies. Unfortunately, likelihood-based learning and inference is hampered by the intractability of computing the normalizing constant. This paper considers an alternative scoring rule to the log-likelihood, which obviates the need to compute the normalizing constant of the distribution. We show that the rule is a quadratic function of the natural parameters.…
A computational approach to maximum likelihood thresholds for colored Gaussian graphical models
Universitat Politècnica de Catalunya - BarcelonaTech (UPC)（加泰罗尼亚理工大学（巴塞罗那科技大学））；Aalto University（阿尔托大学）；Max Planck Institute of Biochemistry（马克斯·普朗克生物化学研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
We study the problem of estimability of means in undirected graphical Gaussian models with symmetry restrictions represented by a colored graph. Following on from previous studies, we partition the variables into sets of vertices whose corresponding means are restricted to being identical. We find a necessary and sufficient condition on the partition to ensure equality between the maximum likelihood and least-squares estimators of the mean.
Stellenbosch University（斯泰伦博斯大学）；National Institute for Theoretical and Computational Sciences（国家理论与计算科学研究所）
Initial work on variational autoencoders assumed independent latent variables with simple distributions. Subsequent work has explored incorporating more complex distributions and dependency structures: including normalizing flows in the encoder network allows latent variables to entangle non-linearly, creating a richer class of distributions for the approximate posterior, and stacking layers of latent variables allows more complex priors to be specified for the generative model. This work explores incorporating arbitrary dependency structures, as specified by Bayesian networks, into VAEs. This…
We study maximum likelihood estimation in Gaussian graphical models from a geometric point of view. An algebraic elimination criterion allows us to find exact lower bounds on the number of observations needed to ensure that the maximum likelihood estimator (MLE) exists with probability one. This is applied to bipartite graphs, grids and colored graphs. We also study the ML degree, and we present the first instance of a graph for which the MLE exists with probability one, even when the number of observations equals the treewidth.
