基于局部坐标编码的高维非线性学习
High Dimensional Nonlinear Learning using Local Coordinate Coding
摘要:
提出了一种新的高维非线性流形上的半监督学习方法,包括无监督基学习和有监督函数学习两个阶段。学习的基提供一组锚点以形成局部坐标系,使得流形上的每个数据点$x$可以通过其附近锚点的线性组合来局部近似,其中线性权重提供$x$的局部坐标编码。我们表明,一个高维的非线性函数可以近似为一个全球性的线性函数相对于这种编码方案,和近似质量是由这样的编码的局部性保证。该方法将复杂的非线性学习问题转化为简单的全局线性学习问题,克服了传统局部学习方法的一些缺点。这项工作还提供了一个理论上的理由,一些生物启发的模型使用稀疏编码的感官数据的经验成功,因为本地编码方案必须足够稀疏。然而,稀疏性并不总是满足局部性条件,因此可能导致次优结果。该方法的属性和性能的经验验证合成数据,手写数字分类,对象识别任务。
英文摘要:
This paper introduces a new method for semi-supervised learning on high dimensional nonlinear manifolds, which includes a phase of unsupervised basis learning and a phase of supervised function learning. The learned bases provide a set of anchor points to form a local coordinate system, such that each data point $x$ on the manifold can be locally approximated by a linear combination of its nearby anchor points, with the linear weights offering a local-coordinate coding of $x$. We show that a high dimensional nonlinear function can be approximated by a global linear function with respect to this coding scheme, and the approximation quality is ensured by the locality of such coding. The method turns a difficult nonlinear learning problem into a simple global linear learning problem, which overcomes some drawbacks of traditional local learning methods. The work also gives a theoretical justification to the empirical success of some biologically-inspired models using sparse coding of sensory data, since a local coding scheme must be sufficiently sparse. However, sparsity does not always satisfy locality conditions, and can thus possibly lead to suboptimal results. The properties and performances of the method are empirically verified on synthetic data, handwritten digit classification, and object recognition tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。