arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.00614cs.LGcs.NAmath.NAstat.ML
收藏

从嵌入在欧氏空间中的低维数据学习的副作用

Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space

The University of Texas at Austin(德克萨斯大学奥斯汀分校);Oden Institute for Computational Engineering and Sciences(奥登计算工程与科学研究所)

Juncai He, Richard Tsai, Rachel Ward

摘要:

低维流形假设假定在许多应用中发现的数据,例如涉及自然图像的数据,(近似)位于嵌入高维欧几里得空间的低维流形上。在这种设置中,典型的神经网络定义了一个函数,该函数将嵌入空间中的有限数量的向量作为输入。然而,人们通常需要考虑在训练分布之外的点处评估优化的网络。本文考虑了训练数据分布在$\mathbb R^d$的线性子空间中的情况。我们得到的学习函数,定义一个神经网络,在横向的子空间的方向上的变化的估计。我们研究了与网络深度和数据流形余维中的噪声相关的潜在正则化效应。由于噪声的存在,我们还提出了训练中的其他副作用。

英文摘要:

The low-dimensional manifold hypothesis posits that the data found in many applications, such as those involving natural images, lie (approximately) on low-dimensional manifolds embedded in a high-dimensional Euclidean space. In this setting, a typical neural network defines a function that takes a finite number of vectors in the embedding space as input. However, one often needs to consider evaluating the optimized network at points outside the training distribution. This paper considers the case in which the training data is distributed in a linear subspace of $\mathbb R^d$. We derive estimates on the variation of the learning function, defined by a neural network, in the direction transversal to the subspace. We study the potential regularization effects associated with the network's depth and noise in the codimension of the data manifold. We also present additional side effects in training due to the presence of noise.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑