arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.02862stat.MLcs.AIcs.LG
收藏

图像数据流形并假设的证明

Verifying the Union of Manifolds Hypothesis for Image Data

University of Waterloo(滑铁卢大学);Layer 6 AI(Layer 6人工智能公司)

Bradley C. A. Brown, Anthony L. Caterini, Brendan Leigh Ross, Jesse C. Cresswell, Gabriel Loaiza-Ganem

摘要:

深度学习在学习高维数据的低维表示方面取得了巨大的成功。如果在感兴趣的数据中没有隐藏的低维结构,这种成功是不可能的;这种存在是由流形假设假定的,该假设指出数据位于低内在维度的未知流形上。在本文中,我们认为,这一假设并没有正确地捕捉到通常存在于图像数据中的低维结构。假设数据位于单个流形上意味着内在维度在整个数据空间中是相同的,并且不允许该空间的子区域具有不同数量的变化因子。为了解决这个问题,我们考虑流形假设的联合,它指出数据位于不同内在维度的流形的不相交的联合上。我们在常用的图像数据集上实证验证了这一假设,发现观察到的数据确实位于一个不连通的集合上,并且内在维度不是恒定的。我们还提供了对流形联合假设在深度学习中的影响的见解,包括监督和无监督,表明设计具有这种结构的归纳偏差的模型可以提高分类和生成建模任务的性能。我们的代码可在https://github.com/layer6ai-labs/UoMH上获得。

英文摘要:

Deep learning has had tremendous success at learning low-dimensional representations of high-dimensional data. This success would be impossible if there was no hidden low-dimensional structure in data of interest; this existence is posited by the manifold hypothesis, which states that the data lies on an unknown manifold of low intrinsic dimension. In this paper, we argue that this hypothesis does not properly capture the low-dimensional structure typically present in image data. Assuming that data lies on a single manifold implies intrinsic dimension is identical across the entire data space, and does not allow for subregions of this space to have a different number of factors of variation. To address this deficiency, we consider the union of manifolds hypothesis, which states that data lies on a disjoint union of manifolds of varying intrinsic dimensions. We empirically verify this hypothesis on commonly-used image datasets, finding that indeed, observed data lies on a disconnected set and that intrinsic dimension is not constant. We also provide insights into the implications of the union of manifolds hypothesis in deep learning, both supervised and unsupervised, showing that designing models with an inductive bias for this structure improves performance across classification and generative modelling tasks. Our code is available at https://github.com/layer6ai-labs/UoMH.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑