arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1910.11552cs.LGstat.ML
收藏

用于分类的正则化权值直接确定的Gegenbauer神经网络

A Gegenbauer Neural Network with Regularized Weights Direct Determination for Classification

School of Automation Science and Engineering(自动化科学与工程学院);South China University of Technology(华南理工大学)

Jie He, Tao Chen, Zhijun Zhang

摘要:

单隐层前馈神经网络(SLFN)在模式分类问题中有着广泛的应用,但传统的迭代梯度学习算法存在速度慢、性能差的问题。虽然著名的极限学习机(ELM)已经成功地解决了收敛速度慢的问题,但它仍然存在输入权重和偏差随机分配带来的计算鲁棒性问题。因此,为了克服上述问题,在本文中,一种新型的神经网络的基础上Gegenbauer正交多项式,称为GNN,构建和研究。该模型可以克服ELM的计算鲁棒性问题,同时仍然具有相当的结构简单性和逼近能力。在此基础上,我们提出了一个正则化的权重直接确定(R-WDD)基于等式约束优化,以确定最佳的输出权重。R-WDD的目标是最小化网络的经验风险和结构风险,从而降低过拟合风险,提高泛化能力。这导致我们最终使用R-WDD的GNN,这是一种用于二进制和多类分类问题的统一学习机制。最后,如在各种比较实验中所验证的,与最小二乘支持向量机(LS-SVM)、具有高斯核的ELM相比,具有R-WDD的GNN倾向于具有相当的(甚至更好的)泛化性能、计算可扩展性和效率以及分类鲁棒性。

英文摘要:

Single-hidden layer feed forward neural networks (SLFNs) are widely used in pattern classification problems, but a huge bottleneck encountered is the slow speed and poor performance of the traditional iterative gradient-based learning algorithms. Although the famous extreme learning machine (ELM) has successfully addressed the problems of slow convergence, it still has computational robustness problems brought by input weights and biases randomly assigned. Thus, in order to overcome the aforementioned problems, in this paper, a novel type neural network based on Gegenbauer orthogonal polynomials, termed as GNN, is constructed and investigated. This model could overcome the computational robustness problems of ELM, while still has comparable structural simplicity and approximation capability. Based on this, we propose a regularized weights direct determination (R-WDD) based on equality-constrained optimization to determine the optimal output weights. The R-WDD tends to minimize the empirical risks and structural risks of the network, thus to lower the risk of over fitting and improve the generalization ability. This leads us to a the final GNN with R-WDD, which is a unified learning mechanism for binary and multi-class classification problems. Finally, as is verified in the various comparison experiments, GNN with R-WDD tends to have comparable (or even better) generalization performances, computational scalability and efficiency, and classification robustness, compared to least square support vector machine (LS-SVM), ELM with Gaussian kernel.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑