We study multivariate Gaussian models that are described by linear conditions on the concentration matrix. We compute the maximum likelihood (ML) degrees of these models. That is, we count the critical points of the likelihood function over a linear space of symmetric matrices. We obtain new formulae for the ML degree, one via Schubert calculus, and another using Segre classes from intersection theory. We settle the case of codimension one models, and characterize the degenerate case when the ML degree is zero.
We study the maximum likelihood degree of linear concentration models in algebraic statistics. We relate the geometry of the reciprocal variety to that of semidefinite programming. We show that the Zariski closure in the Grassmanian of the set of linear spaces that do not attain their maximal possible maximum likelihood degree coincides with the Zariski closure of the set of linear spaces defining a projection with non-closed image of the positive semidefinite cone. In particular, this shows that this closure is a union of coisotropic hypersurfaces.
For general data, the number of complex solutions to the likelihood equations is constant and this number is called the (maximum likelihood) ML-degree of the model. In this article, we describe the special locus of data for which the likelihood equations have a solution in the model's singular locus.
We show that algebraic varieties with maximum likelihood degree one are exactly the images of reduced A-discriminantal varieties under monomial maps with finite fibers. The maximum likelihood estimator corresponding to such a variety is Kapranov's Horn uniformization. This extends Kapranov's characterization of A-discriminantal hypersurfaces to varieties of arbitrary codimension.
We consider statistical models arising from the common set of solutions to a sparse polynomial system with general coefficients. The maximum likelihood degree counts the number of critical points of the likelihood function restricted to the model. We prove the maximum likelihood degree of a sparse polynomial system is determined by its Newton polytopes and equals the mixed volume of a related Lagrange system of equations.
We establish connections between invariant theory and maximum likelihood estimation for discrete statistical models. We show that norm minimization over a torus orbit is equivalent to maximum likelihood estimation in log-linear models. We use notions of stability under a torus action to characterize the existence of the maximum likelihood estimate, and discuss connections to scaling algorithms.
Maximum likelihood estimation is a fundamental computational problem in statistics. In this note, we give a bound for the maximum likelihood degree of algebraic statistical models for discrete data. As usual, such models are identified with special very affine varieties. Using earlier work of Franecki and Kapranov, we prove that the maximum likelihood degree is always less or equal to the signed intersection-cohomology Euler characteristic. We construct counterexamples to a bound in terms of the usual Euler characteristic conjectured by Huh and Sturmfels.
In this article, we investigate Maximum Likelihood Estimation with tools from Tropical Geometry and Bernstein--Sato theory. We investigate the critical points of very affine varieties and study their asymptotic behavior. We relate these asymptotics to particular rays in the tropical variety as well as to Bernstein--Sato ideals and give a connection to Maximum Likelihood Estimation in Statistics.
We give an explicit formula for the reciprocal maximum likelihood degree of Brownian motion tree models. To achieve this, we connect them to certain toric (or log-linear) models, and express the Brownian motion tree model of an arbitrary tree as a toric fiber product of star tree models.
Maximum likelihood estimation in statistics leads to the problem of maximizing a product of powers of polynomials. We study the algebraic degree of the critical equations of this optimization problem. This degree is related to the number of bounded regions in the corresponding arrangement of hypersurfaces, and to the Euler characteristic of the complexified complement. Under suitable hypotheses, the maximum likelihood degree equals the top Chern class of a sheaf of logarithmic differential forms. Exact formulae in terms of degrees and Newton polytopes are given for polynomials with generic coe…
We study the problem of maximum likelihood estimation for $3$-dimensional linear spaces of $3\times 3$ symmetric matrices from the point of view of algebraic statistics where we view these nets of conics as linear concentration or linear covariance models of Gaussian distributions on $\mathbb{R}^3$. In particular, we study the reciprocal surfaces of nets of conics which are rational surfaces in $\mathbb{P}^5$. We show that the reciprocal surfaces are projections from the Veronese surface and determine their intersection with the polar nets. This geometry explains the maximum likelihood degrees…
Maximum likelihood estimation (MLE) is a fundamental computational problem in statistics. In this paper, MLE for statistical models with discrete data is studied from an algebraic statistics viewpoint. A reformulation of the MLE problem in terms of dual varieties and conormal varieties will be given. With this description, the dual likelihood equations and the dual MLE problem are defined. We show that solving the dual MLE problem yields solutions to the MLE problem, so we can solve the MLE problem without ever determining the defining equations of the model.
We study the maximum likelihood (ML) degree of linear concentration models in algebraic statistics. We relate it to an intersection problem on the variety of complete quadrics. This allows us to provide an explicit, basic, albeit of high computational complexity, formula for the ML-degree. The variety of complete quadrics is an exact analog for symmetric matrices of the permutohedron variety for the diagonal matrices.
We uncover connections between maximum likelihood estimation in statistics and norm minimization over a group orbit in invariant theory. We focus on Gaussian transformation families, which include matrix normal models and Gaussian graphical models given by transitive directed acyclic graphs. We use stability under group actions to characterize boundedness of the likelihood, and existence and uniqueness of the maximum likelihood estimate. Our approach reveals promising consequences of the interplay between invariant theory and statistics. In particular, existing scaling algorithms from statisti…
The Expectation-Maximization (EM) algorithm is routinely used for the maximum likelihood estimation in the latent class analysis. However, the EM algorithm comes with no guarantees of reaching the global optimum. We study the geometry of the latent class model in order to understand the behavior of the maximum likelihood estimator. In particular, we characterize the boundary stratification of the binary latent class model with a binary hidden variable. For small models, such as for three binary observed variables, we show that this stratification allows exact computation of the maximum likelih…
Correlation matrices are standardized covariance matrices. They form an affine space of symmetric matrices defined by setting the diagonal entries to one. We study the geometry of maximum likelihood estimation for this model and linear submodels that encode additional symmetries. We also consider the problem of minimizing two closely related functions of the covariance matrix: the Stein's loss and the symmetrized Stein's loss. Unlike the Gaussian log-likelihood these two functions are convex and hence admit a unique positive definite optimum. Some of our results hold for general affine covaria…
In this paper, we study sample size thresholds for maximum likelihood estimation for tensor normal models. Given the model parameters and the number of samples, we determine whether, almost surely, (1) the likelihood function is bounded from above, (2) maximum likelihood estimates (MLEs) exist, and (3) MLEs exist uniquely. We obtain a complete answer for both real and complex models. One consequence of our results is that almost sure boundedness of the log-likelihood function guarantees almost sure existence of an MLE. Our techniques are based on invariant theory and castling transforms.
Maximum likelihood estimation is a fundamental optimization problem in statistics. We study this problem on manifolds of matrices with bounded rank. These represent mixtures of distributions of two independent discrete random variables. We determine the maximum likelihood degree for a range of determinantal varieties, and we apply numerical algebraic geometry to compute all critical points of their likelihood functions. This led to the discovery of maximum likelihood duality between matrices of complementary ranks, a result proved subsequently by Draisma and Rodriguez.
In this paper, we study the log-likelihood function and Maximum Likelihood Estimate (MLE) for the matrix normal model for both real and complex models. We describe the exact number of samples needed to achieve (almost surely) three conditions, namely a bounded log-likelihood function, existence of MLEs, and uniqueness of MLEs. As a consequence, we observe that almost sure boundedness of log-likelihood function guarantees almost sure existence of an MLE, thereby proving a conjecture of Drton, Kuriki and Hoff. The main tools we use are from the theory of quiver representations, in particular, re…
In a recent paper, Hauenstein, Sturmfels, and the second author discovered a conjectural bijection between critical points of the likelihood function on the complex variety of matrices of rank r and critical points on the complex variety of matrices of co-rank r-1. In this paper, we prove that conjecture for rectangular matrices and for symmetric matrices, as well as a variant for skew-symmetric matrices. To appear in International Mathematics Research Notices.
In algebraic statistics, the maximum likelihood degree of a statistical model is the number of complex critical points of its log-likelihood function. A priori knowledge of this number is useful for applying techniques of numerical algebraic geometry to the maximum likelihood estimation problem. We compute the maximum likelihood degree of a generic two-dimensional subspace of the space of $n\times n$ Gaussian covariance matrices. We use the intersection theory of plane curves to show that this number is $2n-3$.
Maximum likelihood degree of a projective variety is the number of critical points of a general likelihood function. In this note, we compute the Maximum likelihood degree of Fermat hypersurfaces. We give a formula of the Maximum likelihood degree in terms of the constants $β_{μ, ν}$, which is defined to be the number of complex solutions to the system of equations $z_1^ν=z_2^ν=\cdots=z_μ^ν=1$ and $z_1+\cdots +z_μ+1=0$.
We give a numerical algorithm computing Euler obstruction functions using maximum likelihood degrees. The maximum likelihood degree is a well-studied property of a variety in algebraic statistics and computational algebraic geometry. In this article we use this degree to give a new way to compute Euler obstruction functions. We define the maximum likelihood obstruction function and show how it coincides with the Euler obstruction function. With this insight, we are able to bring new tools of computational algebraic geometry to study Euler obstruction functions.
The minimum number of observations such that the maximum likelihood estimator in a Gaussian graphical model exists with probability one is called the maximum likelihood threshold of the underlying graph G. The natural algebraic relaxation is the generic completion rank introduced by Uhler. We show that the maximum likelihood threshold and the generic completion rank behave in the same way under clique sums, which gives us large families of graphs on which these invariants coincide. On the other hand we determine both invariants for complete bipartite graphs $K_{m,n}$ and show that for some cho…
We study the critical points of the likelihood function over the Fermat hypersurface. This problem is related to one of the main problems in statistical optimization: maximum likelihood estimation. The number of critical points over a projective variety is a topological invariant of the variety and is called maximum likelihood degree. We provide closed formulas for the maximum likelihood degree of any Fermat curve in the projective plane and of Fermat hypersurfaces of degree 2 in any projective space. Algorithmic methods to compute the ML degree of a generic Fermat hypersurface are developed t…
