arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1507.01892cs.LGphysics.data-an
收藏

一种基于双层神经网络的线性稀疏编码方法

A linear approach for sparse coding by a two-layer neural network

Ghent University(根特大学);University of Naples Federico II(那不勒斯费德里科二世大学)

Alessandro Montalto, Giovanni Tessitore, Roberto Prevete

摘要:

最近文献中提出了许多通过特征变换将非线性分类问题转化为线性分类问题的方法。这些方法主要包括稀疏编码方法和深度神经网络。然而,这些方法中的许多方法需要在呈现看不见的数据输入向量时重复应用学习过程,或者涉及使用大量的参数和超参数,这些参数和超参数必须通过交叉验证来选择,从而显著增加运行时间。在本文中,我们提出并实验研究了一种新的方法,以克服这两种限制的目的。所提出的方法利用了一个线性自联想网络(称为SCNN),只有一个隐藏层。该架构与要最小化的特定误差函数的组合使得能够学习计算稀疏码的线性编码器,该稀疏码被证明尽可能类似于通过重新训练神经网络获得的稀疏编码。重要的是,SCNN的线性度和误差函数的选择允许在学习阶段实现减少的运行时间。建议的架构进行评估的基础上,两个标准的机器学习任务。它的性能进行了比较,最近提出的非线性自联想神经网络。总体结果表明,线性编码器可以有利地用于获得稀疏的数据表示的上下文中的机器学习问题,提供了一个适当的误差函数在学习阶段。

英文摘要:

Many approaches to transform classification problems from non-linear to linear by feature transformation have been recently presented in the literature. These notably include sparse coding methods and deep neural networks. However, many of these approaches require the repeated application of a learning process upon the presentation of unseen data input vectors, or else involve the use of large numbers of parameters and hyper-parameters, which must be chosen through cross-validation, thus increasing running time dramatically. In this paper, we propose and experimentally investigate a new approach for the purpose of overcoming limitations of both kinds. The proposed approach makes use of a linear auto-associative network (called SCNN) with just one hidden layer. The combination of this architecture with a specific error function to be minimized enables one to learn a linear encoder computing a sparse code which turns out to be as similar as possible to the sparse coding that one obtains by re-training the neural network. Importantly, the linearity of SCNN and the choice of the error function allow one to achieve reduced running time in the learning phase. The proposed architecture is evaluated on the basis of two standard machine learning tasks. Its performances are compared with those of recently proposed non-linear auto-associative neural networks. The overall results suggest that linear encoders can be profitably used to obtain sparse data representations in the context of machine learning problems, provided that an appropriate error function is used during the learning phase.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑