arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1811.01464cs.LGstat.ML
收藏

非线性嵌入的本征普适测度

Intrinsic Universal Measurements of Non-linear Embeddings

CSIRO Data61(澳大利亚联邦科学与工业研究组织Data61);The Australian National University(澳大利亚国立大学)

Ke Sun

摘要:

机器学习中的一个基本问题是找到从低维潜在空间$\mathcal{Y}$到高维观测空间$\mathcal{X}$的映射$f$。诸如深度神经网络之类的现代工具能够表示一般的非线性映射。一个学习者可以很容易地找到一个映射,完全符合所有的观察。然而,这样的映射通常不被认为是好的,因为它不够简单并且可能过拟合。如何定义简单性?我们试图对非线性映射$f$所施加的信息量做出一个正式的定义。直观地,我们测量的拉回几何和潜在空间的内在几何之间的局部差异。我们的定义是基于信息几何,是独立的经验观察,也没有具体的参数化。我们证明了它的基本性质,并讨论了相关的机器学习方法的关系。

英文摘要:

A basic problem in machine learning is to find a mapping $f$ from a low dimensional latent space $\mathcal{Y}$ to a high dimensional observation space $\mathcal{X}$. Modern tools such as deep neural networks are capable to represent general non-linear mappings. A learner can easily find a mapping which perfectly fits all the observations. However, such a mapping is often not considered as good, because it is not simple enough and can overfit. How to define simplicity? We try to make a formal definition on the amount of information imposed by a non-linear mapping $f$. Intuitively, we measure the local discrepancy between the pullback geometry and the intrinsic geometry of the latent space. Our definition is based on information geometry and is independent of the empirical observations, nor specific parameterizations. We prove its basic properties and discuss relationships with related machine learning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑