样本外延拓的数学分析
Mathematical Analysis on Out-of-Sample Extensions
摘要：
令$X=\mathbf{X}\cup\mathbf{Z}$为$\mathbb{R}^D$中的数据集，其中$\mathbf{X}$为训练集，$\mathbf{Z}$为测试集。已经开发了许多基于核方法的无监督学习算法来为给定的训练集$Φ: \mathbf{X} \to \mathbb{R}^d$（$d\ll D$）提供降维（DR）嵌入，该训练集将高维数据$\mathbf{X}$映射到其低维特征表示$\mathbf{Y}=Φ(\mathbf{X})$。然而，这些算法不能直接产生测试集$\mathbf{Z}$的DR。样本外扩展方法使用现有嵌入$Φ$的扩展来提供$\mathbf{Z}$的DR，而不是重新计算整个集合$X$的DR嵌入。在各种样外DR扩展方法中，基于Nyström近似的方法是非常有吸引力的。许多文献提出了这种超拓算法，并通过数值实验证明了它们的有效性。然而，DR扩展的数学理论仍需要进一步考虑。利用再生核希尔伯特空间（RKHS）理论，对样本外DR扩张算子进行了初步的数学分析。它将样本外DR扩展运算符视为$\mathbf{X}$上定义的RKHS上的恒等式的扩展。然后Nyström型DR扩展原来是一个正交投影。本文还给出了DR延拓的精确条件，并给出了延拓的误差估计。
英文摘要：
Let $X=\mathbf{X}\cup\mathbf{Z}$ be a data set in $\mathbb{R}^D$, where $\mathbf{X}$ is the training set and $\mathbf{Z}$ is the test one. Many unsupervised learning algorithms based on kernel methods have been developed to provide dimensionality reduction (DR) embedding for a given training set $Φ: \mathbf{X} \to \mathbb{R}^d$ ( $d\ll D$) that maps the high-dimensional data $\mathbf{X}$ to its low-dimensional feature representation $\mathbf{Y}=Φ(\mathbf{X})$. However, these algorithms do not straightforwardly produce DR of the test set $\mathbf{Z}$. An out-of-sample extension method provides DR of $\mathbf{Z}$ using an extension of the existent embedding $Φ$, instead of re-computing the DR embedding for the whole set $X$. Among various out-of-sample DR extension methods, those based on Nyström approximation are very attractive. Many papers have developed such out-of-extension algorithms and shown their validity by numerical experiments. However, the mathematical theory for the DR extension still need further consideration. Utilizing the reproducing kernel Hilbert space (RKHS) theory, this paper develops a preliminary mathematical analysis on the out-of-sample DR extension operators. It treats an out-of-sample DR extension operator as an extension of the identity on the RKHS defined on $\mathbf{X}$. Then the Nyström-type DR extension turns out to be an orthogonal projection. In the paper, we also present the conditions for the exact DR extension and give the estimate for the error of the extension.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
