arXiv 1406.22402014-06-09math.STstat.MLstat.TH

Feature Selection For High-Dimensional Clustering

Larry Wasserman, Martin Azizyan, Aarti Singh

备注：: 11 pages, 2 figures

英文摘要：

We present a nonparametric method for selecting informative features in high-dimensional clustering problems. We start with a screening step that uses a test for multimodality. Then we apply kernel density estimation and mode clustering to the selected features. The output of the method consists of a list of relevant features, and cluster assignments. We provide explicit bounds on the error rate of the resulting clustering. In addition, we provide the first error bounds on mode based clustering.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Modal clustering asymptotics with applications to bandwidth selection

Density-based clustering relies on the idea of linking groups to some specific features of the probability distribution underlying the data. The reference to a true, yet unknown, population structure allows to frame the clustering problem in a standard inferential setting, where the concept of ideal population clustering is defined as the partition induced by the true density function. The nonparametric formulation of this approach, known as modal clustering, draws a correspondence between the groups and the domains of attraction of the density modes. Operationally, a nonparametric density est…

2019-01-22stat.MEstat.ML

Simultaneous Dimensionality and Complexity Model Selection for Spectral Graph Clustering

Naval Surface Warfare Center Dahlgren Division（美国海军水面战中心达尔格伦分部）；Johns Hopkins University（约翰霍普金斯大学）

Our problem of interest is to cluster vertices of a graph by identifying underlying community structure. Among various vertex clustering approaches, spectral clustering is one of the most popular methods because it is easy to implement while often outperforming more traditional clustering algorithms. However, there are two inherent model selection problems in spectral clustering, namely estimating both the embedding dimension and number of clusters. This paper attempts to address the issue by establishing a novel model selection framework specifically for vertex clustering on graphs under a st…

2020-09-22cs.LGstat.ML

A model selection approach for clustering a multinomial sequence with non-negative factorization

We consider a problem of clustering a sequence of multinomial observations by way of a model selection criterion. We propose a form of a penalty term for the model selection procedure. Our approach subsumes both the conventional AIC and BIC criteria but also extends the conventional criteria in a way that it can be applicable also to a sequence of sparse multinomial observations, where even within a same cluster, the number of multinomial trials may be different for different observations. In addition, as a preliminary estimation step to maximum likelihood estimation, and more generally, to ma…

2015-08-14stat.ML

Bayesian Model Selection for Change Point Detection and Clustering

KTH Royal Institute of Technology（KTH皇家理工学院）

We address the new problem of estimating a piece-wise constant signal with the purpose of detecting its change points and the levels of clusters. Our approach is to model it as a nonparametric penalized least square model selection on a family of models indexed over the collection of partitions of the design points and propose a computationally efficient algorithm to approximately solve it. Statistically, minimizing such a penalized criterion yields an approximation to the maximum a posteriori probability (MAP) estimator. The criterion is then analyzed and an oracle inequality is derived using…

2019-12-03cs.LGmath.STstat.ML

Clustering High-dimensional Data via Feature Selection

High-dimensional clustering analysis is a challenging problem in statistics and machine learning, with broad applications such as the analysis of microarray data and RNA-seq data. In this paper, we propose a new clustering procedure called Spectral Clustering with Feature Selection (SC-FS), where we first obtain an initial estimate of labels via spectral clustering, then select a small fraction of features with the largest R-squared with these labels, i.e., the proportion of variation explained by group labels, and conduct clustering again using selected features. Under mild conditions, we pro…

2022-10-27stat.ME

Eigen selection in spectral clustering: a theory guided practice

Based on a Gaussian mixture type model , we derive an eigen selection procedure that improves the usual spectral clustering in high-dimensional settings. Concretely, we derive the asymptotic expansion of the spiked eigenvalues under eigenvalue multiplicity and eigenvalue ratio concentration results, giving rise to the first theory-backed eigen selection procedure in spectral clustering. The resulting eigen-selected spectral clustering (ESSC) algorithm enjoys better stability and compares favorably against canonical alternatives. We demonstrate the advantages of ESSC using extensive simulation …

2020-06-07math.STstat.TH

更多相关论文收起更多论文

Discriminative variable selection for clustering with the sparse Fisher-EM algorithm

The interest in variable selection for clustering has increased recently due to the growing need in clustering high-dimensional data. Variable selection allows in particular to ease both the clustering and the interpretation of the results. Existing approaches have demonstrated the efficiency of variable selection for clustering but turn out to be either very time consuming or not sparse enough in high-dimensional spaces. This work proposes to perform a selection of the discriminative variables by introducing sparsity in the loading matrix of the Fisher-EM algorithm. This clustering method has…

2012-04-10math.STstat.MEstat.TH

Combining clustering of variables and feature selection using random forests

Standard approaches to tackle high-dimensional supervised classification problem often include variable selection and dimension reduction procedures. The novel methodology proposed in this paper combines clustering of variables and feature selection. More precisely, hierarchical clustering of variables procedure allows to build groups of correlated variables in order to reduce the redundancy of information and summarizes each group by a synthetic numerical variable. Originality is that the groups of variables (and the number of groups) are unknown a priori. Moreover the clustering approach use…

2018-11-06math.STstat.TH

Finite mixture regression: A sparse variable selection by model selection for clustering

We consider a finite mixture of Gaussian regression model for high- dimensional data, where the number of covariates may be much larger than the sample size. We propose to estimate the unknown conditional mixture density by a maximum likelihood estimator, restricted on relevant variables selected by an 1-penalized maximum likelihood estimator. We get an oracle inequality satisfied by this estimator with a Jensen-Kullback-Leibler type loss. Our oracle inequality is deduced from a general model selection theorem for maximum likelihood estimators with a random model collection. We can derive the …

2014-09-04math.STstat.TH

基于模型的多等位基因位点聚类分析

Model-Based Clustering using multi-allelic loci data with loci selection

我们提出了一个基于模型的聚类（MBC）方法结合位点选择使用多等位基因位点的遗传数据。将基因座选择问题转化为模型选择问题，并将竞争模型与贝叶斯信息准则进行比较。由此产生的程序选择聚类位点的子集，聚类数，估计每个聚类的比例和每个聚类内的等位基因频率。我们证明了所选择的模型收敛于真实模型的概率在一个单一的现实假设下，作为样本的大小趋于无穷大。用c++编程语言实现了该方法MixMoGenD（Mixture Model using Genetic Data）。模拟数据集上的数值实验进行了突出的建议位点选择过程的兴趣。

2008-12-30math.STstat.APstat.TH

Estimation and Model Selection for Model-Based Clustering with the Conditional Classification Likelihood

The Integrated Completed Likelihood (ICL) criterion has been proposed by Biernacki et al. (2000) in the model-based clustering framework to select a relevant number of classes and has been used by statisticians in various application areas. A theoretical study of this criterion is proposed. A contrast related to the clustering objective is introduced: the conditional classification likelihood. This yields an estimator and a model selection criteria class. The properties of these new procedures are studied and ICL is proved to be an approximation of one of these criteria. We oppose these result…

2012-05-31math.STstat.TH

Clustering and Model Selection via Penalized Likelihood for Different-sized Categorical Data Vectors

In this study, we consider unsupervised clustering of categorical vectors that can be of different size using mixture. We use likelihood maximization to estimate the parameters of the underlying mixture model and a penalization technique to select the number of mixture components. Regardless of the true distribution that generated the data, we show that an explicit penalty, known up to a multiplicative constant, leads to a non-asymptotic oracle inequality with the Kullback-Leibler divergence on the two sides of the inequality. This theoretical result is illustrated by a document clustering app…

2017-09-07math.STstat.TH

Clustering and variable selection for categorical multivariate data

This article investigates unsupervised classification techniques for categorical multivariate data. The study employs multivariate multinomial mixture modeling, which is a type of model particularly applicable to multilocus genotypic data. A model selection procedure is used to simultaneously select the number of components and the relevant variables. A non-asymptotic oracle inequality is obtained, leading to the proposal of a new penalized maximum likelihood criterion. The selected model proves to be asymptotically consistent under weak assumptions on the true probability underlying the obser…

2014-03-08math.STstat.TH

具有聚类特定对角协方差阵和分组变量的惩罚模型聚类

Penalized model-based clustering with cluster-specific diagonal covariance matrices and grouped variables

聚类分析是在许多新兴领域如微阵列数据分析中最广泛使用的统计工具之一。对于微阵列和其他高维数据，许多噪声变量的存在可能掩盖潜在的聚类结构。因此，通过变量选择去除噪声变量是必要的。对于同时进行变量选择和参数估计的问题，现有的基于模型的聚类分析中的惩罚似然方法都假设聚类之间的对角协方差阵是公共的，但这在实际中可能并不成立。为了分析高维数据，特别是那些样本量相对较低的数据，本文介绍了一种新的方法，该方法在更一般的情况下使用特定于聚类的（对角）协方差阵来缩小方差和均值。此外，通过包含或排除一组变量来选择分组变量是通过特定形式的惩罚来允许的，这有助于结合主题知识，例如用于疾病亚型发现的聚类微阵列样品中的基因功能。为了实现，期望最大化算法推导出参数估计，其中的M-步骤清楚地表明收缩和阈值的影响。数值例子，包括应用于急性白血病亚型发现与微阵列基因表达数据，提供证明所提出的方法的实用性和优势。

2008-03-26math.STstat.MLstat.TH

通过聚类进行多元极值的估计与阶选择

On estimation and order selection for multivariate extremes via clustering

本文提出带惩罚的简化平均轮廓宽度来一致选择多元极值谱测度的聚类阶，并分析聚类估计的大偏差性质及重尾因子模型参数估计。

2025-02-19math.STstat.MEstat.TH

模型基聚类中非随机缺失下变量选择的统一框架

A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random

Faculty of Mathematics and Computer Science, University of Science, Ho Chi Minh City, Vietnam（越南科技大学数学与计算机科学学院）；Vietnam National University Ho Chi Minh City, Vietnam（越南胡志明市国家大学）；School of Information and Communications Technology, Hanoi University of Science and Technology, Ha Noi, Vietnam（河内科学技术大学信息与通信技术学院）；ARC Centre of Excellence for the Mathematical Analysis of Cellular Systems（细胞系统数学分析卓越中心）；School of Mathematical Sciences, Queensland University of Technology, Brisbane City, Australia（昆士兰科技大学数学科学学院）

本文提出一个统一框架，同时解决模型基聚类中非随机缺失数据与变量选择问题，通过引入数据驱动惩罚矩阵和缺失机制建模，实现渐近与选择一致性。

2025-11-04cs.LGmath.STstat.AP