局部欠采样数据的本征维数估计
Intrinsic dimension estimation for locally undersampled data
摘要:
高维数据在当代科学中无处不在,找到压缩它们的方法是机器学习的主要目标之一。给定一个位于高维空间(原则上是数百到数千维)的数据集,将其投影到低维流形上通常是有用的,而不会丢失信息。确定这种流形的最小维数是一个具有挑战性的问题,在文献中称为固有维数估计(IDE)。传统上,大多数IDE算法要么基于多尺度主成分分析(PCA),要么基于相关维的概念(更一般地说,基于k-最近邻距离)。这些方法受到严重的维数灾难的不同影响。特别地,现有算法中没有一个可以在极端局部欠采样机制中提供准确的ID估计,即在流形的任何局部补丁中的样本数量小于数据集的ID(或与数据集的ID相同的数量级)的极限中。在这里,我们引入了一个新的ID估计,利用流形的切线空间的简单属性来克服这些缺点。该方法基于全关联积分,突破了关联维数估计的小半径限制。我们的估计器解决了极端欠采样问题,与其他方法难以解决。基于这一认识,我们探讨了多尺度推广的算法。我们证明了它能够(i)识别数据集中的多个维度,以及(ii)提供对极端弯曲流形的ID的准确估计。特别是,我们测试的高对比度图像的全局变换生成的流形上的方法,相关的不变对象识别,并认为是最先进的ID估计的挑战。
英文摘要:
High-dimensional data are ubiquitous in contemporary science and finding methods to compress them is one of the primary goals of machine learning. Given a dataset lying in a high-dimensional space (in principle hundreds to several thousands of dimensions), it is often useful to project it onto a lower-dimensional manifold, without loss of information. Identifying the minimal dimension of such manifold is a challenging problem known in the literature as intrinsic dimension estimation (IDE). Traditionally, most IDE algorithms are either based on multiscale principal component analysis (PCA) or on the notion of correlation dimension (and more in general on k-nearest-neighbors distances). These methods are affected, in different ways, by a severe curse of dimensionality. In particular, none of the existing algorithms can provide accurate ID estimates in the extreme locally undersampled regime, i.e. in the limit where the number of samples in any local patch of the manifold is less than (or of the same order of) the ID of the dataset. Here we introduce a new ID estimator that leverages on simple properties of the tangent space of a manifold to overcome these shortcomings. The method is based on the full correlation integral, going beyond the limit of small radius used for the estimation of the correlation dimension. Our estimator alleviates the extreme undersampling problem, intractable with other methods. Based on this insight, we explore a multiscale generalization of the algorithm. We show that it is capable of (i) identifying multiple dimensionalities in a dataset, and (ii) providing accurate estimates of the ID of extremely curved manifolds. In particular, we test the method on manifolds generated from global transformations of high-contrast images, relevant for invariant object recognition and considered a challenge for state-of-the-art ID estimators.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。