arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1603.02763cs.LGcs.CGstat.ML
收藏

megaman:Manifold Learning with Millions of points

megaman: Manifold Learning with Millions of points

University of Washington(华盛顿大学)

James McQueen, Marina Meila, Jacob VanderPlas, Zhongyue Zhang

摘要:

流形学习是一类寻求高维数据的低维非线性表示的算法。因此,流形学习算法,至少在理论上,最适用于高维数据和样本大小,使准确估计的流形。尽管如此,大多数现有的流形学习实现不是特别可扩展的。在这里,我们提出了一个Python包,实现了各种各样的流形学习算法在一个模块化和可扩展的方式,使用快速近似邻居搜索和快速稀疏特征分解。该软件包结合了流形学习的理论进展,如无偏拉普拉斯估计和黎曼度量方法的嵌入失真估计。在基准测试中,即使在单核台式计算机上,我们的代码也能在几分钟内嵌入数百万个数据点,仅需200分钟就能嵌入斯隆数字巡天的星系光谱的主要样本-由3750维的60万个样本组成-这是以前不可能完成的任务。

英文摘要:

Manifold Learning is a class of algorithms seeking a low-dimensional non-linear representation of high-dimensional data. Thus manifold learning algorithms are, at least in theory, most applicable to high-dimensional data and sample sizes to enable accurate estimation of the manifold. Despite this, most existing manifold learning implementations are not particularly scalable. Here we present a Python package that implements a variety of manifold learning algorithms in a modular and scalable fashion, using fast approximate neighbors searches and fast sparse eigendecompositions. The package incorporates theoretical advances in manifold learning, such as the unbiased Laplacian estimator and the estimation of the embedding distortion by the Riemannian metric method. In benchmarks, even on a single-core desktop computer, our code embeds millions of data points in minutes, and takes just 200 minutes to embed the main sample of galaxy spectra from the Sloan Digital Sky Survey --- consisting of 0.6 million samples in 3750-dimensions --- a task which has not previously been possible.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑