Parameter Estimation in Gaussian Mixture Models with Malicious Noise, without Balanced Mixing Coefficients

用于跨组聚类的分层shot-noise Cox过程混合模型

Hierarchical shot-noise Cox process mixtures for clustering across groups

本文提出分层shot-noise Cox过程混合模型，通过核共享中心实现跨组聚类与密度估计的平衡，并开发条件MCMC算法，在模拟和星系数据上优于分层Dirichlet过程。

On the Role of Channel Capacity in Learning Gaussian Mixture Models

This paper studies the sample complexity of learning the $k$ unknown centers of a balanced Gaussian mixture model (GMM) in $\mathbb{R}^d$ with spherical covariance matrix $σ^2\mathbf{I}$. In particular, we are interested in the following question: what is the maximal noise level $σ^2$, for which the sample complexity is essentially the same as when estimating the centers from labeled measurements? To that end, we restrict attention to a Bayesian formulation of the problem, where the centers are uniformly distributed on the sphere $\sqrt{d}\mathcal{S}^{d-1}$. Our main results characterize the e…

On shrinkage estimation for balanced loss functions

The estimation of a multivariate mean $θ$ is considered under natural modifications of balanced loss function of the form: (i) $ω\, ρ(\|δ-δ_0\|^2) + (1-ω) \, ρ(\|δ-θ\|^2) $, and (ii) $\ell \left( ω\, \|δ-δ_0\|^2 + (1-ω) \, \|δ-θ\|^2 \right)\,$, where $δ_0$ is a target estimator of $γ(θ)$. After briefly reviewing known results for original balanced loss with identity $ρ$ or $\ell$, we provide, for increasing and concave $ρ$ and $\ell$ which also satisfy a completely monotone property, Baranchik-type estimators of $θ$ which dominate the benchmark $δ_0(X)=X$ for $X$ either distributed as multivar…

Uniform Convergence Rates for Maximum Likelihood Estimation under Two-Component Gaussian Mixture Models

We derive uniform convergence rates for the maximum likelihood estimator and minimax lower bounds for parameter estimation in two-component location-scale Gaussian mixture models with unequal variances. We assume the mixing proportions of the mixture are known and fixed, but make no separation assumption on the underlying mixture components. A phase transition is shown to exist in the optimal parameter estimation rate, depending on whether or not the mixture is balanced. Key to our analysis is a careful study of the dependence between the parameters of location-scale Gaussian mixture models, a…

On shrinkage estimation of a spherically symmetric distribution for balanced loss functions

We consider the problem of estimating the mean vector $θ$ of a $d$-dimensional spherically symmetric distributed $X$ based on balanced loss functions of the forms: {\bf (i)} $ωρ(\|\de-\de_{0}\|^{2}) +(1-ω)ρ(\|\de - θ\|^{2})$ and {\bf (ii)} $\ell\left(ω\|\de - \de_{0}\|^{2} +(1-ω)\|\de - θ\|^{2}\right)$, where $δ_0$ is a target estimator, and where $ρ$ and $\ell$ are increasing and concave functions. For $d\geq 4$ and the target estimator $δ_0(X)=X$, we provide Baranchik-type estimators that dominate $δ_0(X)=X$ and are minimax. The findings represent extensions of those of Marchand \& Strawderm…

Efficient Clustering for Stretched Mixtures: Landscape and Optimality

This paper considers a canonical clustering problem where one receives unlabeled samples drawn from a balanced mixture of two elliptical distributions and aims for a classifier to estimate the labels. Many popular methods including PCA and k-means require individual components of the mixture to be somewhat spherical, and perform poorly when they are stretched. To overcome this issue, we propose a non-convex program seeking for an affine transform to turn the data into a one-dimensional point cloud concentrating around $-1$ and $1$, after which clustering becomes easy. Our theoretical contribut…

Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures

We analyze the connection between minimizers with good generalizing properties and high local entropy regions of a threshold-linear classifier in Gaussian mixtures with the mean squared error loss function. We show that there exist configurations that achieve the Bayes-optimal generalization error, even in the case of unbalanced clusters. We explore analytically the error-counting loss landscape in the vicinity of a Bayes-optimal solution, and show that the closer we get to such configurations, the higher the local entropy, implying that the Bayes-optimal solution lays inside a wide flat regio…