非线性嵌入快速学习的部分Hessian策略
Partial-Hessian Strategies for Fast Learning of Nonlinear Embeddings
摘要：
随机邻域嵌入（SNE）和相关的非线性流形学习算法实现了相似数据的高质量低维表示，但训练非常慢。我们提出了一个通用的配方嵌入算法，包括SNE和其他现有的算法，并研究它们与谱方法和图拉普拉斯算子的关系。这使我们能够定义几个部分Hessian优化策略，表征其全局和局部收敛性，并根据经验对其进行评估。我们实现了两个数量级的加速比现有的训练方法与策略（我们称之为频谱方向），几乎没有增加任何开销的梯度，但简单，可扩展，适用于几个现有的和未来的嵌入算法。
英文摘要：
Stochastic neighbor embedding (SNE) and related nonlinear manifold learning algorithms achieve high-quality low-dimensional representations of similarity data, but are notoriously slow to train. We propose a generic formulation of embedding algorithms that includes SNE and other existing algorithms, and study their relation with spectral methods and graph Laplacians. This allows us to define several partial-Hessian optimization strategies, characterize their global and local convergence, and evaluate them empirically. We achieve up to two orders of magnitude speedup over existing training methods with a strategy (which we call the spectral direction) that adds nearly no overhead to the gradient and yet is simple, scalable and applicable to several existing and future embedding algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
