局部成分分析
Local Component Analysis
摘要:
核密度估计Parzen窗是一种流行的密度估计方法,可用于离群点检测或聚类。对于多变量数据,其性能严重依赖于内核中使用的度量。大多数早期的工作都集中在只学习内核的带宽(即,标量乘法因子)。在本文中,我们建议通过期望最小化(EM)过程学习一个完整的欧几里德度量,这可以被视为邻域分量分析(NCA)的无监督对应。为了避免在高维中使用完全非参数密度估计的过拟合,我们还考虑了半参数Gaussian-Parzen密度模型,其中一些变量通过联合高斯密度建模,而其他变量通过Parzen窗口建模。对于这两个模型,EM导致简单的封闭形式的更新的基础上矩阵求逆和特征值分解。我们的经验表明,我们的方法导致密度估计具有更高的测试似然比自然竞争的方法,和该指标可以使用在大多数无监督学习技术,依赖于这样的指标,如谱聚类或流形学习方法。最后,我们提出了一个随机近似方案,允许使用这种方法在一个大规模的设置。
英文摘要:
Kernel density estimation, a.k.a. Parzen windows, is a popular density estimation method, which can be used for outlier detection or clustering. With multivariate data, its performance is heavily reliant on the metric used within the kernel. Most earlier work has focused on learning only the bandwidth of the kernel (i.e., a scalar multiplicative factor). In this paper, we propose to learn a full Euclidean metric through an expectation-minimization (EM) procedure, which can be seen as an unsupervised counterpart to neighbourhood component analysis (NCA). In order to avoid overfitting with a fully nonparametric density estimator in high dimensions, we also consider a semi-parametric Gaussian-Parzen density model, where some of the variables are modelled through a jointly Gaussian density, while others are modelled through Parzen windows. For these two models, EM leads to simple closed-form updates based on matrix inversions and eigenvalue decompositions. We show empirically that our method leads to density estimators with higher test-likelihoods than natural competing methods, and that the metrics may be used within most unsupervised learning techniques that rely on such metrics, such as spectral clustering or manifold learning methods. Finally, we present a stochastic approximation scheme which allows for the use of this method in a large-scale setting.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。