arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1412.6749cs.CVcs.AIcs.NEmath.OCstat.ML
收藏

SENNS:用于特征提取的稀疏提取神经网络

SENNS: Sparse Extraction Neural NetworkS for Feature Extraction

Obafemi Awolowo University(奥巴费米·阿沃洛沃大学)

Abdulrahman Oladipupo Ibraheem

摘要:

通过借鉴优化理论,人工神经网络(ANN),图嵌入和稀疏表示的思想,我开发了一种新的技术,称为SENNS(稀疏提取神经网络),旨在解决特征提取问题。所提出的方法使用(优选地深)ANN用于将输入属性向量投影到输出空间,其中对于属于不同类的向量,成对距离被最大化,但是对于属于同一类的向量,成对距离被最小化,同时对ANN输出执行稀疏性。从投影产生的向量然后可以用作任何选择的分类器中的特征。在数学上,我将所提出的方法表述为目标函数的最小化,在ANN输出空间中,该目标函数可以被解释为属于不同类别的输出向量之间成对距离平方和的负因子,加上属于相同类别的输出向量之间成对距离平方和的正因子,加上稀疏性和权重衰减项。为了通过梯度下降推导出最小化目标函数的算法,我使用多变量版本的链式规则来获得函数相对于ANN权重和偏置的偏导数,并发现每个所需的偏导数可以表示为六项之和。事实证明,这六项中的四项可以使用标准反向传播算法计算;第五项可以通过对标准反向传播算法的轻微修改来计算;而第六项可以通过简单的算术计算。最后,我提出了实验的阿拉伯数字和字母的阿拉伯语语料库,CMU PIE数据库的面孔,MNIST数字数据库,和其他标准的机器学习数据库。

英文摘要:

By drawing on ideas from optimisation theory, artificial neural networks (ANN), graph embeddings and sparse representations, I develop a novel technique, termed SENNS (Sparse Extraction Neural NetworkS), aimed at addressing the feature extraction problem. The proposed method uses (preferably deep) ANNs for projecting input attribute vectors to an output space wherein pairwise distances are maximized for vectors belonging to different classes, but minimized for those belonging to the same class, while simultaneously enforcing sparsity on the ANN outputs. The vectors that result from the projection can then be used as features in any classifier of choice. Mathematically, I formulate the proposed method as the minimisation of an objective function which can be interpreted, in the ANN output space, as a negative factor of the sum of the squares of the pair-wise distances between output vectors belonging to different classes, added to a positive factor of the sum of squares of the pair-wise distances between output vectors belonging to the same classes, plus sparsity and weight decay terms. To derive an algorithm for minimizing the objective function via gradient descent, I use the multi-variate version of the chain rule to obtain the partial derivatives of the function with respect to ANN weights and biases, and find that each of the required partial derivatives can be expressed as a sum of six terms. As it turns out, four of those six terms can be computed using the standard back propagation algorithm; the fifth can be computed via a slight modification of the standard backpropagation algorithm; while the sixth one can be computed via simple arithmetic. Finally, I propose experiments on the ARABASE Arabic corpora of digits and letters, the CMU PIE database of faces, the MNIST digits database, and other standard machine learning databases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑