Many statistical and machine learning approaches rely on pairwise distances between data points. The choice of distance metric has a fundamental impact on performance of these procedures, raising questions about how to appropriately calculate distances. When data points are real-valued vectors, by far the most common choice is the Euclidean distance. This article is focused on the problem of how to better calculate distances taking into account the intrinsic geometry of the data, assuming data are concentrated near an unknown subspace or manifold. The appropriate geometric distance corresponds to the length of the shortest path along the manifold, which is the geodesic distance. When the manifold is unknown, it is challenging to accurately approximate the geodesic distance. Current algorithms are either highly complex, and hence often impractical to implement, or based on simple local linear approximations and shortest path algorithms that may have inadequate accuracy. We propose a simple and general alternative, which uses pieces of spheres, or spherelets, to locally approximate the unknown subspace and thereby estimate the geodesic distance through paths over spheres. Theory is developed showing lower error for many manifolds, with applications in clustering, conditional density estimation and mean regression. The conclusion is supported through multiple simulation examples and real data sets.
Minimax-optimal semi-supervised regression on unknown manifolds
Weizmann Institute of Science(魏茨曼科学研究所)
我们考虑半监督回归时,预测变量是从一个未知的流形。解决这个问题的一个简单的两步方法是:(i)使用标记和未标记的实例来估计任何一对点之间的流形测地线距离;以及(ii)基于这些距离估计应用k最近邻回归量。我们证明了足够多的未标记的点,这种简单的方法测地线kNN回归达到最佳的有限样本极大极小界的均方误差,如果已知的流形。此外,我们展示了如何可以有效地实现这种方法,只需要O(k N log N)的操作来估计所有N个标记和未标记点的回归函数。我们在两个具有流形结构的数据集上说明了这种方法:使用WiFi指纹的室内定位和面部姿态估计。在这两种情况下,测地线kNN比流行的拉普拉斯特征向量回归更准确,更快。
Near optimal sample complexity for matrix and tensor normal models via geodesic convexity
Department of Mathematics, Massachusetts Institute of Technology(麻省理工学院数学系);Cheriton School of Computer Science, University of Waterloo(滑铁卢大学丘成桐计算机科学学院);Computer Science Department, University of British Columbia(不列颠哥伦比亚大学计算机科学系);Ludwig-Maximilians-Universität München(慕尼黑路德维希-马克西米利安大学)
Functional Principal Component Analysis for Manifold-Indexed Data
The Institute for Data Innovation in Science, Seoul National University(首尔大学数据创新科学研究所);Department of Statistics and School of Transdisciplinary Innovations, Seoul National University(首尔大学统计学院与跨学科创新学院)