Xiaohui Chen, Yun Yang
We determine the information-theoretic cutoff value on separation of cluster centers for exact recovery of cluster labels in a $K$-component Gaussian mixture model with equal cluster sizes. Moreover, we show that a semidefinite programming (SDP) relaxation of the $K$-means clustering method achieves such sharp threshold for exact recovery without assuming the symmetry of cluster centers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Mixture models are widely used to fit complex and multimodal datasets. In this paper we study mixtures with high dimensional sparse latent parameter vectors and consider the problem of support recovery of those vectors. While parameter learning in mixture models is well-studied, the sparsity constraint remains relatively unexplored. Sparsity of parameter vectors is a natural constraint in variety of settings, and support recovery is a major step towards parameter estimation. We provide efficient algorithms for support recovery that have a logarithmic sample complexity dependence on the dimensi…
We study the community detection problem on a Gaussian mixture model, in which vertices are divided into $k\geq 2$ distinct communities. The major difference in our model is that the intensities for Gaussian perturbations are different for different entries in the observation matrix, and we do not assume that every community has the same number of vertices. We explicitly find the threshold for the exact recovery of the maximum likelihood estimation. Applications include the community detection on hypergraphs.
Exact Recovery of Community Detection in dependent Gaussian Mixture Models
研究在依赖且异质高斯噪声的高斯混合模型中社区检测的精确恢复，提出在奇异协方差矩阵情况下MLE的约束二次优化问题，并推导出在社区大小未知或已知时的精确恢复条件。
Exact Sign Recovery for PCA Connectivity Analysis in Mixture Models
本文研究PCA连通性分析中的精确符号恢复，提出充分条件并给出渐近精确截止值，无需条件数有界。
In this paper, we propose a generalized scale mixture family of distributions, namely the Power Exponential Scale Mixture (PESM) family, to model the sparsity inducing priors currently in use for sparse signal recovery (SSR). We show that the successful and popular methods such as LASSO, Reweighted $\ell_1$ and Reweighted $\ell_2$ methods can be formulated in an unified manner in a maximum a posteriori (MAP) or Type I Bayesian framework using an appropriate member of the PESM family as the sparsity inducing prior. In addition, exploiting the natural hierarchical framework induced by the PESM f…
This paper studies the problem of clustering in the two-component Gaussian mixture model where the centers are separated by $2Δ$ for some $Δ>0$. We characterize the exact phase transition threshold, given by $$ \barΔ_n^{2} = σ^{2}\left(1 + \sqrt{1+\frac{2p}{n\log{n}}} \right)\log{n}, $$ such that perfect recovery of the communities is possible with high probability if $Δ\ge(1+\varepsilon)\bar Δ_n$, and impossible if $Δ\le (1-\varepsilon)\bar Δ_n$ for any constant $\varepsilon>0$. This implies an elbow effect at a critical dimension $p^{*}=n\log{n}$. We present a non-asymptotic lower bound for …
Despite the ubiquity of kernel-based clustering, surprisingly few statistical guarantees exist beyond settings that consider strong structural assumptions on the data generation process. In this work, we take a step towards bridging this gap by studying the statistical performance of kernel-based clustering algorithms under non-parametric mixture models. We provide necessary and sufficient separability conditions under which these algorithms can consistently recover the underlying true clustering. Our analysis provides guarantees for kernel clustering approaches without structural assumptions …
Signal Recovery Using a Spiked Mixture Model
本文提出尖峰混合模型及其EM算法，用于在低信噪比情况下提升信号恢复性能，应用于生物医学和计算机视觉领域，优于传统方法。
In this article, we study the binomial mixture model under the regime that the binomial size $m$ can be relatively large compared to the sample size $n$. This project is motivated by the GeneFishing method (Liu et al., 2019), whose output is a combination of the parameter of interest and the subsampling noise. To tackle the noise in the output, we utilize the observation that the density of the output has a U shape and model the output with the binomial mixture model under a U shape constraint. We first analyze the estimation of the underlying distribution F in the binomial mixture model under…
Information Gap and Feasibility-Aware Inference in Binomial Logistic Mixtures
研究二项逻辑混合模型中混合检测与标签恢复之间的信息差距，提出基于后验熵惩罚的可行性感知推断方法，避免误导性成分选择并改善后验标签概率校准。
Private estimation algorithms for stochastic block models and mixture models
本文提出高维私有估计通用工具，实现随机块模型恢复和球面高斯混合中心恢复，统计保证接近非私有算法，且为首个高效差分私有算法。
One-Bit Clustering for Two Component Sub-Gaussian Mixture Models
提出首个用于两个分量次高斯混合模型的单比特聚类方法，使用抖动量化器每样本仅取一位，在温和条件下Lloyd算法变体实现指数级衰减的误分类率，且精确恢复条件仅比未量化数据的最优阈值多对数因子。
Mixture and separation of log-concave measures
研究双组分对数凹混合模型，探讨给定样本时各组分分布参数的精确恢复程度，识别出混合分布的两种情况，提出用二次分类器分离组分分布的方法。
Gaussian Mixture Model with unknown diagonal covariances via continuous sparse regularization
本文提出利用Beurling-LASSO方法估计具有未知对角协方差的高斯混合模型，同时估计组件数量和参数，并在理论上证明了非渐近恢复保证和近参数收敛率。
We consider the problem of estimating the discrete clustering structures under the Sub-Gaussian Mixture Model. Our main results establish a hidden integrality property of a semidefinite programming (SDP) relaxation for this problem: while the optimal solution to the SDP is not integer-valued in general, its estimation error can be upper bounded by that of an idealized integer program. The error of the integer program, and hence that of the SDP, are further shown to decay exponentially in the signal-to-noise ratio. In addition, we show that the SDP relaxation is robust under the semi-random set…
We study the fundamental tradeoffs between statistical accuracy and computational tractability in the analysis of high dimensional heterogeneous data. As examples, we study sparse Gaussian mixture model, mixture of sparse linear regressions, and sparse phase retrieval model. For these models, we exploit an oracle-based computational model to establish conjecture-free computationally feasible minimax lower bounds, which quantify the minimum signal strength required for the existence of any algorithm that is both computationally tractable and statistically accurate. Our analysis shows that there…
We investigate the clustering performances of the relaxed $K$means in the setting of sub-Gaussian Mixture Model (sGMM) and Stochastic Block Model (SBM). After identifying the appropriate signal-to-noise ratio (SNR), we prove that the misclassification error decay exponentially fast with respect to this SNR. These partial recovery bounds for the relaxed $K$means improve upon results currently known in the sGMM setting. In the SBM setting, applying the relaxed $K$means SDP allows to handle general connection probabilities whereas other SDPs investigated in the literature are restricted to the as…
In sensing applications, sensors cannot always measure the latent quantity of interest at the required resolution, sometimes they can only acquire a blurred version of it due the sensor's transfer function. To recover latent signals when only noisy mixed measurements of the signal are available, we propose the Gaussian process mixture of measurements (GPMM), which models the latent signal as a Gaussian process (GP) and allows us to perform Bayesian inference on such signal conditional to a set of noisy mixture of measurements. We describe how to train GPMM, that is, to find the hyperparameters…
Non-Gaussian mixture models are gaining increasing attention for mixture model-based clustering particularly when dealing with data that exhibit features such as skewness and heavy tails. Here, such a mixture distribution is presented, based on the multivariate normal inverse Gaussian (MNIG) distribution. For parameter estimation of the mixture, a Bayesian approach via Gibbs sampler is used; for this, a novel approach to simulate univariate generalized inverse Gaussian random variables and matrix generalized inverse Gaussian random matrices is provided. The proposed algorithm will be applied t…
The horseshoe prior, defined as a half Cauchy scale mixture of normal, provides a state of the art approach to Bayesian sparse signal recovery. We provide a new representation of the horseshoe density as a scale mixture of the Laplace density, explicitly identifying the mixing measure. Using the celebrated Bernstein--Widder theorem and a result due to Bochner, our representation immediately establishes the complete monotonicity of the horseshoe density and strong concavity of the corresponding penalty. Consequently, the equivalence between local linear approximation and expectation--maximizati…
Pair Correlation Factor and the Sample Complexity of Gaussian Mixtures
本文提出配对相关因子（PCF）作为更准确衡量高斯混合模型样本复杂度的几何量，并给出均匀球形情形下改进样本复杂度界限的算法。
Individual-heterogeneous sub-Gaussian Mixture Models
本文提出个体异质子高斯混合模型，通过为每个观测分配异质性参数，有效捕捉实际应用中的异质性。该模型基于谱方法，在弱分离条件下可精确恢复真实聚类标签，优于传统高斯混合模型。
In regression models, predictor variables with inherent ordering, such as tumor staging ranging and ECOG performance status, are commonly seen in medical settings. Statistically, it may be difficult to determine the functional form of an ordinal predictor variable. Often, such a variable is dichotomized based on whether it is above or below a certain cutoff. Other methods conveniently treat the ordinal predictor as a continuous variable and assume a linear relationship with the outcome. However, arbitrarily choosing a method may lead to inaccurate inference and treatment. In this paper, we pro…
The trimming scheme with a prefixed cutoff portion is known as a method of improving the robustness of statistical models such as multivariate Gaussian mixture models (MG- MMs) in small scale tests by alleviating the impacts of outliers. However, when this method is applied to real- world data, such as noisy speech processing, it is hard to know the optimal cut-off portion to remove the outliers and sometimes removes useful data samples as well. In this paper, we propose a new method based on measuring the dispersion degree (DD) of the training data to avoid this problem, so as to realise auto…
Mixture-of-Finite-Mixtures Wishart Model for Clustering Covariance Matrices with an Application to Brain Functional Connectivity
提出MFM-Wishart贝叶斯模型，结合威沙特混合分量与有限混合先验，实现协方差矩阵数据的聚类与簇数联合推断，并应用于婴儿睡眠功能连接聚类。
The question of how to determine the number of independent latent factors (topics) in mixture models such as Latent Dirichlet Allocation (LDA) is of great practical importance. In most applications, the exact number of topics is unknown, and depends on the application and the size of the data set. Bayesian nonparametric methods can avoid the problem of topic number selection, but they can be impracticably slow for large sample sizes and are subject to local optima. We develop a guaranteed procedure for topic number recovery that does not necessitate learning the model's latent parameters befor…
Multi-kernel spectral clustering: Entrywise eigenvector perturbation bounds and exact recovery
针对高维多尺度数据的单一带宽核谱聚类不足问题，提出多核谱聚类方法，建立逐元素特征向量扰动界，证明其结合近似K-means可高概率实现数据精确恢复。
Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures
本文针对过参数化高斯混合模型设定，证明随机初始化的梯度EM在适度过参数化下可全局收敛到真实模型，是EM在$m=2$外的首个全局收敛结果。
With rapid development of techniques to measure brain activity and structure, statistical methods for analyzing modern brain-imaging play an important role in the advancement of science. Imaging data that measure brain function are usually multivariate time series and are heterogeneous across both imaging sources and subjects, which lead to various statistical and computational challenges. In this paper, we propose a group-based method to cluster a collection of multivariate time series via a Bayesian mixture of smoothing splines. Our method assumes each multivariate time series is a mixture o…
Mixtures of multivariate normal inverse Gaussian (MNIG) distributions can be used to cluster data that exhibit features such as skewness and heavy tails. However, for cluster analysis, using a traditional finite mixture model framework, either the number of components needs to be known $a$-$priori$ or needs to be estimated $a$-$posteriori$ using some model selection criterion after deriving results for a range of possible number of components. However, different model selection criteria can sometimes result in different number of components yielding uncertainty. Here, an infinite mixture model…
Growth mixture models are an important tool for detecting group structure in repeated measures data. Unlike traditional clustering methods, they explicitly model the repeat measurements on observations, and the statistical framework they are based on allows for model selection methods to be used to select the number of clusters. However, the basic growth mixture model makes the assumption that all of the measurements in the data have grouping information/separate the clusters. In other clustering contexts, it has been shown that including non-clustering variables in clustering procedures can l…
Riemannian Gradient Descent for Gaussian Mixture Models with unknown diagonal covariances
本文针对未知分量数与对角协方差的高斯混合模型，提出结合锥粒子梯度下降与黎曼梯度下降的求解方法，理论证明指数收敛，实验表明其比EM算法对分量数过指定更鲁棒。
Local linear convergence of gradient methods for overparameterized Gaussian mixtures
针对过参数化高斯混合模型，提出一种交替使用短梯度步和长Polyak步的方法，实现局部线性收敛速率，克服了过参数化导致的慢收敛问题。
In this paper, we consider a parametric density contamination model. We work with a sample of i.i.d. data with a common density, $f^\star =(1-λ^\star) ϕ+ λ^\star ϕ(.-μ^\star)$, where the shape $ϕ$ is assumed to be known. We establish the optimal rates of convergence for the estimation of the mixture parameters $(λ^\star,μ^\star)$. In particular, we prove that the classical parametric rate $1/\sqrt{n}$ cannot be reached when at least one of these parameters is allowed to tend to $0$ with $n$.
Mixture-of-experts Wishart model for covariance matrices with an application to Cancer drug screening
本文提出了一种混合专家Wishart模型，用于分析异质协方差数据，并应用于癌症药物敏感性分析。
Time-partitioning Gaussian Process Mixture Model
本研究提出TPGPMM模型，用于识别时间序列的持续状态并建模非平稳依赖，通过贝叶斯非参数方法实现，在应用于金融数据时可识别与重大事件对齐的状态，且存在可解释性与预测准确性的权衡。
Robust Deep Mixture Models
该研究提出一种基于通路级共享尺度混合构造的鲁棒深度混合模型，采用随机期望最大化算法估计参数，在重尾及污染场景下聚类性能优于深度高斯混合模型，可识别异质潜在结构并降低异常观测影响。
Laplace and skew-Laplace approximations for Dirichlet process mixture posterior density
本文研究了狄利克雷过程混合模型后验密度估计中拉普拉斯和斜拉普拉斯近似方法的性能，通过数值实验比较了标准拉普拉斯近似、修正斜拉普拉斯近似和切片采样基准，发现斜拉普拉斯近似在复杂密度结构中能显著提升后验恢复精度。
A mixture model for subtype identification in the context of disease progression modeling
本文提出了一种概率混合模型，用于在疾病进展建模中识别不同亚型，通过时间空间变异聚类，提高分类准确性和参数估计精度。
A Model-Based Clustering Approach for Bounded Data Using Transformation-Based Gaussian Mixture Models
本文提出一种基于范围幂变换与高斯混合模型的有界数据概率聚类方法，并引入 NCE 度量不确定性，实验表明其簇恢复与可解释性优于现有方法。
We develop a novel peak detection algorithm for the analysis of comprehensive two-dimensional gas chromatography time-of-flight mass spectrometry (GC$\times$GC-TOF MS) data using normal-exponential-Bernoulli (NEB) and mixture probability models. The algorithm first performs baseline correction and denoising simultaneously using the NEB model, which also defines peak regions. Peaks are then picked using a mixture of probability distribution to deal with the co-eluting peaks. Peak merging is further carried out based on the mass spectral similarities among the peaks within the same peak group. T…
Estimating Gaussian mixtures using sparse polynomial moment systems
本文利用多面体几何研究高斯混合模型的矩可识别性，证明一般高斯k-混合模型可由前3k+2个矩识别，并提出路径数随维度线性增长的同伦算法用于高维参数恢复。
We study the non-convex optimization landscape for maximum likelihood estimation in the discrete orbit recovery model with Gaussian noise. This model is motivated by applications in molecular microscopy and image processing, where each measurement of an unknown object is subject to an independent random rotation from a rotational group. Equivalently, it is a Gaussian mixture model where the mixture centers belong to a group orbit. We show that fundamental properties of the likelihood landscape depend on the signal-to-noise ratio and the group structure. At low noise, this landscape is "benign"…
Moment Identifiability of Homoscedastic Gaussian Mixtures
本文研究了同方差高斯混合的矩可识别性，通过分析矩流形和割线流形，确定了在不同成分数情况下参数恢复的方法和理论依据。
On the Geometry of Separation in Finite Gaussian Mixtures
研究最小成分分离对有限高斯混合参数估计收敛速率的影响，通过构建基于Hellinger下界的统一几何框架，揭示了分离复杂度由成分空间配置驱动，并获得了依赖分离的收敛速率。
Mixture model for designs in high dimensional regression and the LASSO
针对实际中设计矩阵常不满足低相干性导致LASSO估计进入慢速率区域的问题，提出列向量服从高斯混合模型的设计矩阵模型，仅要求混合中心矩阵低相干，即可实现与不相关设计精度相近的Xβ估计。
Sparse recovery from linear Gaussian measurements has been the subject of much investigation since the breaktrough papers \cite{CRT:IEEEIT06} and \cite{donoho2006compressed} on Compressed Sensing. Application to sparse vectors and sparse matrices via least squares penalized with sparsity promoting norms is now well understood using tools such as Gaussian mean width, statistical dimension and the notion of descent cones \cite{tropp2014convex} \cite{Vershynin:ArXivEstimation14}. Extention of these ideas to low rank tensor recovery is starting to enjoy considerable interest due to its many potent…
Sharp and Adaptive Cluster Recovery in Slowly Mixing Gaussian Hidden Markov Models
本文研究缓慢混合高斯隐马尔可夫模型中的簇恢复，揭示统计难度集中于罕见转移点，提出自适应多项式时间算法，达到尖锐恢复速率，有效复杂度由潜在转移数而非样本量决定。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。