Multilayer Perceptron with Functional Inputs: an Inverse Regression Approach

Estimating the Number of Components in a Mixture of Multilayer Perceptrons

BIC criterion is widely used by the neural-network community for model selection tasks, although its convergence properties are not always theoretically established. In this paper we will focus on estimating the number of components in a mixture of multilayer perceptrons and proving the convergence of the BIC criterion in this frame. The penalized marginal-likelihood for mixture models and hidden Markov models introduced by Keribin (2000) and, respectively, Gassiat (2002) is extended to mixtures of multilayer perceptrons for which a penalized-likelihood criterion is proposed. We prove its conv…

Efficient Estimation of Multidimensional Regression Model using Multilayer Perceptrons

This work concerns the estimation of multidimensional nonlinear regression models using multilayer perceptrons (MLPs). The main problem with such models is that we need to know the covariance matrix of the noise to get an optimal estimator. However, we show in this paper that if we choose as the cost function the logarithm of the determinant of the empirical error covariance matrix, then we get an asymptotically optimal estimator. Moreover, under suitable assumptions, we show that this cost function leads to a very simple asymptotic law for testing the number of parameters of an identifiable M…

Consistent estimation of the architecture of multilayer perceptrons

We consider regression models involving multilayer perceptrons (MLP) with one hidden layer and a Gaussian noise. The estimation of the parameters of the MLP can be done by maximizing the likelihood of the model. In this framework, it is difficult to determine the true number of hidden units using an information criterion, like the Bayesian information criteria (BIC), because the information matrix of Fisher is not invertible if the number of hidden units is overestimated. Indeed, the classical theoretical justification of information criteria relies entirely on the invertibility of this matrix…

Efficient Estimation of Multidimensional Regression Model with Multilayer Perceptron

This work concerns estimation of multidimensional nonlinear regression models using multilayer perceptron (MLP). The main problem with such model is that we have to know the covariance matrix of the noise to get optimal estimator. however we show that, if we choose as cost function the logarithm of the determinant of the empirical error covariance matrix, we get an asymptotically optimal estimator.

Asymptotic law of likelihood ratio for multilayer perceptron models

We consider regression models involving multilayer perceptrons (MLP) with one hidden layer and a Gaussian noise. The data are assumed to be generated by a true MLP model and the estimation of the parameters of the MLP is done by maximizing the likelihood of the model. When the number of hidden units of the true model is known, the asymptotic distribution of the maximum likelihood estimator (MLE) and the likelihood ratio (LR) statistic is easy to compute and converge to a $χ^2$ law. However, if the number of hidden unit is over-estimated the Fischer information matrix of the model is singular a…

宽多层感知机的理论压缩界限

Theoretical Compression Bounds for Wide Multilayer Perceptrons

本文提出了一种随机贪心算法，用于宽多层感知机的剪枝和量化，证明了其在保持性能的同时实现网络压缩的理论界限。