Multilayer neural networks set the current state of the art for many technical classification problems. But, these networks are still, essentially, black boxes in terms of analyzing them and predicting their performance. Here, we develop a statistical theory for the one-layer perceptron and show that it can predict performances of a surprisingly large variety of neural networks with different architectures. A general theory of classification with perceptrons is developed by generalizing an existing theory for analyzing reservoir computing models and connectionist models for symbolic reasoning known as vector symbolic architectures. Our statistical theory offers three formulas leveraging the signal statistics with increasing detail. The formulas are analytically intractable, but can be evaluated numerically. The description level that captures maximum details requires stochastic sampling methods. Depending on the network model, the simpler formulas already yield high prediction accuracy. The quality of the theory predictions is assessed in three experimental settings, a memorization task for echo state networks (ESNs) from reservoir computing literature, a collection of classification datasets for shallow randomly connected networks, and the ImageNet dataset for deep convolutional neural networks. We find that the second description level of the perceptron theory can predict the performance of types of ESNs, which could not be described previously. The theory can predict deep multilayer neural networks by being applied to their output layer. While other methods for prediction of neural networks performance commonly require to train an estimator model, the proposed theory requires only the first two moments of the distribution of the postsynaptic sums in the output neurons. The perceptron theory compares favorably to other methods that do not rely on training an estimator model.
Breast cancer is the most frequently reported cancer type among the women around the globe and beyond that it has the second highest female fatality rate among all cancer types. Despite all the progresses made in prevention and early intervention, early prognosis and survival prediction rates are still unsatisfactory. In this paper, we propose a novel type of perceptron called L-Perceptron which outperforms all the previous supervised learning methods by reaching 97.42 \% and 98.73 \% in terms of accuracy and sensitivity, respectively in Wisconsin Breast Cancer dataset. Experimental results on…
We propose a focus of attention mechanism to speed up the Perceptron algorithm. Focus of attention speeds up the Perceptron algorithm by lowering the number of features evaluated throughout training and prediction. Whereas the traditional Perceptron evaluates all the features of each example, the Attentive Perceptron evaluates less features for easy to classify examples, thereby achieving significant speedups and small losses in prediction accuracy. Focus of attention allows the Attentive Perceptron to stop the evaluation of features at any interim point and filter the example. This creates an…
Behavior prediction plays an essential role in both autonomous driving systems and Advanced Driver Assistance Systems (ADAS), since it enhances vehicle's awareness of the imminent hazards in the surrounding environment. Many existing lane change prediction models take as input lateral or angle information and make short-term (< 5 seconds) maneuver predictions. In this study, we propose a longer-term (5~10 seconds) prediction model without any lateral or angle information. Three prediction models are introduced, including a logistic regression model, a multilayer perceptron (MLP) model, and a r…
National University of Sciences & Technology（国家科学与技术大学）；Manchester Metropolitan University（曼彻斯特城市大学）
With advanced data analytical techniques, efforts for more accurate decision support systems for disease prediction are on rise. Surveys by World Health Organization (WHO) indicate a great increase in number of diabetic patients and related deaths each year. Early diagnosis of diabetes is a major concern among researchers and practitioners. The paper presents an application of \textit{Automatic Multilayer Perceptron }which\textit{ }is combined with an outlier detection method \textit{Enhanced Class Outlier Detection using distance based algorithm }to create a prediction framework named as Enha…
University of Stuttgart（斯图加特大学）；Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所）
Artificial neural networks (NNs) have become the de facto standard in machine learning. They allow learning highly nonlinear transformations in a plethora of applications. However, NNs usually only provide point estimates without systematically quantifying corresponding uncertainties. In this paper a novel approach towards fully Bayesian NNs is proposed, where training and predictions of a perceptron are performed within the Bayesian inference framework in closed-form. The weights and the predictions of the perceptron are considered Gaussian random variables. Analytical expressions for predict…
Mathematical Institute of the S.A.S.A（塞尔维亚科学与艺术学院数学研究所）；University of Calabria（卡拉布里亚大学）
Global demographic and economic changes have a critical impact on the total energy consumption, which is why demographic and economic parameters have to be taken into account when making predictions about the energy consumption. This research is based on the application of a multiple linear regression model and a neural network model, in particular multilayer perceptron, for predicting the energy consumption. Data from five Balkan countries has been considered in the analysis for the period 1995-2014. Gross domestic product, total number of population, and CO2 emission were taken as predictor …
Delhi Technological University（德里理工大学）；CHRIST (Deemed to be University)（克里斯大学（被视为大学））
There are numerous models of quantum neural networks that have been applied to variegated problems such as image classification, pattern recognition etc.Quantum inspired algorithms have been relevant for quite awhile. More recently, in the NISQ era, hybrid quantum classical models have shown promising results. Multi-feature regression is common problem in classical machine learning. Hence we present a comparative analysis of continuous variable quantum neural networks (Variational circuits) and quantum backpropagating multi layer perceptron (QBMLP). We have chosen the contemporary problem of p…
Kalman Kando Faculty of Electrical Engineering, Obuda University（欧布达大学卡尔曼·坎多电气工程学院）；John von Neumann Faculty of Informatics, Obuda University（欧布达大学约翰·冯·诺依曼信息学院）
The accurate prediction of the solar Diffuse Fraction (DF), sometimes called the Diffuse Ratio, is an important topic for solar energy research. In the present study, the current state of Diffuse Irradiance research is discussed and then three robust, Machine Learning (ML) models, are examined using a large dataset (almost 8 years) of hourly readings from Almeria, Spain. The ML models used herein, are a hybrid Adaptive Network-based Fuzzy Inference System (ANFIS), a single Multi-Layer Perceptron (MLP) and a hybrid Multi-Layer Perceptron-Grey Wolf Optimizer (MLP-GWO). These models were evaluate…
The de facto algorithm for training the back pass of a feedforward neural network is backpropagation (BP). The use of almost-everywhere differentiable activation functions made it efficient and effective to propagate the gradient backwards through layers of deep neural networks. However, in recent years, there has been much research in alternatives to backpropagation. This analysis has largely focused on reaching state-of-the-art accuracy in multilayer perceptrons (MLPs) and convolutional neural networks (CNNs). In this paper, we analyze the stability and similarity of predictions and neurons …
The hypothesis here states that neural network algorithms such as Multi-layer Perceptron (MLP) have higher accuracy in differentiating malicious and semi-structured phishing URLs. Compared to classical machine learning algorithms such as Logistic Regression and Multinomial Naive Bayes, the classical algorithms rely heavily on substantial corpus data training and machine learning experts' domain knowledge to perform complex feature engineering. MLP could perform non-linear separable multi-classes classification and focus less on corpus feature training. In addition, backpropagation weight adjus…
Recently, there have been significant developments in neural networks, which led to the frequent use of neural networks in the physics literature. This work is focused on predicting the masses of exotic hadrons, doubly charmed and bottomed baryons using neural networks trained on meson and baryon masses that are determined by experiments. The original data set has been extended using the recently proposed artificial data augmentation methods. We have observed that the neural network's predictive ability increases with the use of augmented data. The results indicated that data augmentation tech…
The necessity of predictive models in the drug discovery industry cannot be understated. With the sheer volume of potentially useful compounds that are considered for use, it is becoming increasingly computationally difficult to investigate the overlapping interactions between drugs. Understanding this is also important to the layperson who needs to know what they can and cannot mix, especially for those who use recreational drugs - which do not have the same rigorous warnings as prescription drugs. Without access to deterministic, experimental results for every drug combination, other methods…
The significance of heating load (HL) accurate approximation is the primary motivation of this research to distinguish the most efficient predictive model among several neural-metaheuristic models. The proposed models are through synthesizing multi-layer perceptron network (MLP) with ant lion optimization (ALO), biogeography-based optimization (BBO), dragonfly algorithm (DA), evolutionary strategy (ES), invasive weed optimization (IWO), and league champion optimization (LCA) hybrid algorithms. Each ensemble is optimized in terms of the operating population. Accordingly, the ALO-MLP, BBO-MLP, D…
Herein, we have compared the performance of SVM and MLP in emotion recognition using speech and song channels of the RAVDESS dataset. We have undertaken a journey to extract various audio features, identify optimal scaling strategy and hyperparameter for our models. To increase sample size, we have performed audio data augmentation and addressed data imbalance using SMOTE. Our data indicate that optimised SVM outperforms MLP with an accuracy of 82 compared to 75%. Following data augmentation, the performance of both algorithms was identical at ~79%, however, overfitting was evident for the SVM…
Simple Full-Spectrum Correlated k-Distribution Model based on Multilayer Perceptron
Beijing Information Science and Technology University（北京信息科技大学）；Beijing Aerospace Petrochemical EC and EP Technology Corporation Limited（北京航天石化EC和EP技术有限公司）；Beijing Jiaotong University（北京交通大学）
Peking University（北京大学）；Tencent Inc.（腾讯公司）；Beijing Institute of Technology（北京理工大学）
Graph neural networks (GNNs) have achieved great success in many graph-based applications. However, the enormous size and high sparsity level of graphs hinder their applications under industrial scenarios. Although some scalable GNNs are proposed for large-scale graphs, they adopt a fixed $K$-hop neighborhood for each node, thus facing the over-smoothing issue when adopting large propagation depths for nodes within sparse regions. To tackle the above issue, we propose a new GNN architecture -- Graph Attention Multi-Layer Perceptron (GAMLP), which can capture the underlying correlations between…
The percentage of patients hospitalized due to hyponatremia is getting higher. Hyponatremia is the deficiency of sodium electrolyte in the human serum. This deficiency might indulge adverse effects and also associated with longer hospital stay or mortality, if it wasnt actively treated and managed. This work predicts the futuristic sodium levels of patients based on their history of health problems using multilayer perceptron and multivariate linear regression algorithm. This work analyses the patients age, information about other disease such as diabetes, pneumonia, liver-disease, malignancy,…
MLPGradientFlow is a software package to solve numerically the gradient flow differential equation $\dot θ= -\nabla \mathcal L(θ; \mathcal D)$, where $θ$ are the parameters of a multi-layer perceptron, $\mathcal D$ is some data set, and $\nabla \mathcal L$ is the gradient of a loss function. We show numerically that adaptive first- or higher-order integration methods based on Runge-Kutta schemes have better accuracy and convergence speed than gradient descent with the Adam optimizer. However, we find Newton's method and approximations like BFGS preferable to find fixed points (local and global…
National Technical University of Athens（雅典国立技术大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University of Patras（帕特雷大学）
Printed Electronics (PE) exhibits on-demand, extremely low-cost hardware due to its additive manufacturing process, enabling machine learning (ML) applications for domains that feature ultra-low cost, conformity, and non-toxicity requirements that silicon-based systems cannot deliver. Nevertheless, large feature sizes in PE prohibit the realization of complex printed ML circuits. In this work, we present, for the first time, an automated printed-aware software/hardware co-design framework that exploits approximate computing principles to enable ultra-resource constrained printed multilayer per…
Huazhong University of Science and Technology（华中科技大学）；Tongji Medical College（同济医学院）；Central China Normal University（华中师范大学）；Wuhan University（武汉大学）
The number of clinical citations received from clinical guidelines or clinical trials has been considered as one of the most appropriate indicators for quantifying the clinical impact of biomedical papers. Therefore, the early prediction of the clinical citation count of biomedical papers is critical to scientific activities in biomedicine, such as research evaluation, resource allocation, and clinical translation. In this study, we designed a four-layer multilayer perceptron neural network (MPNN) model to predict the clinical citation count of biomedical papers in the future by using 9,822,62…
Machine learning algorithms have been widely used in intrusion detection systems, including Multi-layer Perceptron (MLP). In this study, we proposed a two-stage model that combines the Birch clustering algorithm and MLP classifier to improve the performance of network anomaly multi-classification. In our proposed method, we first apply Birch or Kmeans as an unsupervised clustering algorithm to the CICIDS-2017 dataset to pre-group the data. The generated pseudo-label is then added as an additional feature to the training of the MLP-based classifier. The experimental results show that using Birc…
Uppsala University（乌普萨拉大学）；SICS Computer Systems Laboratory, RISE Research Institutes of Sweden AB（瑞典信息通信技术研究所（RISE）计算机系统实验室（SICS））；Department of Information Technology, Division of Systems and Control, Uppsala University（乌普萨拉大学信息技术系系统与控制部）
Like most learning algorithms, the multilayer perceptrons (MLP) is designed to learn a vector of parameters from data. However, in certain scenarios we are interested in learning structured parameters (predictions) in the form of symmetric positive definite matrices. Here, we introduce a variant of the MLP, referred to as the matrix MLP, that is specialized at learning symmetric positive definite matrices. We also present an application of the model within the context of the variational autoencoder (VAE). Our formulation of the VAE extends the vanilla formulation to the cases where the recogni…
University of Tabriz（大不里士大学）；Urmia University（乌尔米耶大学）；Yazd University（亚兹德大学）；Obuda University（欧布达大学）；Duy Tan University（岘港大学（越南））；Ton Duc Thang University（东德大学）
Wind power as a renewable source of energy, has numerous economic, environmental and social benefits. In order to enhance and control renewable wind power, it is vital to utilize models that predict wind speed with high accuracy. Due to neglecting of requirement and significance of data preprocessing and disregarding the inadequacy of using a single predicting model, many traditional models have poor performance in wind speed prediction. In the current study, for predicting wind speed at target stations in the north of Iran, the combination of a multi-layer perceptron model (MLP) with the Whal…
Mimar Sinan Fine Arts University（米马尔·希南美术大学）；University of Leuven (KU Leuven)（鲁汶大学（KU Leuven））；Yildiz Technical University（耶尔德兹技术大学）
Neural networks have become popular in many fields of science since they serve as promising, reliable and powerful tools. In this work, we study the effect of data augmentation on the predictive power of neural network models for nuclear physics data. We present two different data augmentation techniques, and we conduct a detailed analysis in terms of different depths, optimizers, activation functions and random seed values to show the success and robustness of the model. Using the experimental uncertainties for data augmentation for the first time, the size of the training data set is artific…
Gold Price Prediction Using Long Short-Term Memory and Multi-Layer Perceptron with Gray Wolf Optimizer
College of Interdisciplinary Science and Technology University of Tehran（interdisciplinary 科学与技术学院，德黑兰大学）；School of Science and Technology University of New England（科学与技术学院，新英格兰大学）
Classifying hand-written digits and letters has taken a big leap with the introduction of ConvNets. However, on very constrained hardware the time necessary to train such models would be high. Our main contribution is twofold. First, we extensively test an end-to-end vanilla neural network (MLP) approach in pure numpy without any pre-processing or feature extraction done beforehand. Second, we show that basic data mining operations can significantly improve the performance of the models in terms of computational time, without sacrificing much accuracy. We illustrate our claims on a simpler var…
We utilize Wi-Fi communications from smartphones to predict their mobility mode, i.e. walking, biking and driving. Wi-Fi sensors were deployed at four strategic locations in a closed loop on streets in downtown Toronto. Deep neural network (Multilayer Perceptron) along with three decision tree based classifiers (Decision Tree, Bagged Decision Tree and Random Forest) are developed. Results show that the best prediction accuracy is achieved by Multilayer Perceptron, with 86.52% correct predictions of mobility modes.
Shanghai Jiao Tong University（上海交通大学）；University of Hong Kong（香港大学）；Southern University of Science and Technology（南方科技大学）
Electromigration (EM) is one of the major concerns in the reliability analysis of very large scale integration (VLSI) systems due to the continuous technology scaling. Accurately predicting the time-to-failure of integrated circuits (IC) becomes increasingly important for modern IC design. However, traditional methods are often not sufficiently accurate, leading to undesirable over-design especially in advanced technology nodes. In this paper, we propose an approach using multilayer perceptrons (MLP) to compute stress evolution in the interconnect trees during the void nucleation phase. The av…
Search Multilayer Perceptron-Based Fusion for Efficient and Accurate Siamese Tracking
Institute of Mining Artificial Intelligence, Chinese Institute of Coal Science（采矿人工智能研究所，中国煤炭科学研究院）；Department of Computer Science, City University of Hong Kong（计算机科学系，香港城市大学）；Image Processing Center, School of Astronautics, Beihang University（图像处理中心，航天学院，北航）；State Key Laboratory of Intelligent Coal Mining and Strata Control（智能煤炭开采与岩层控制国家重点实验室）
T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation
Shandong University（山东大学）；City University of Hong Kong（香港城市大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；Texas A&M University（德克萨斯A&M大学）
We present an automatic multilayer power plane generation method to accelerate the design of printed circuit boards (PCB). In PCB design, while automatic solvers have been developed to predict important indicators such as the IR-drop, power integrity, and signal integrity, the generation of the power plane itself still largely relies on laborious manual methods. Our automatic power plane generation approach is based on genetic optimization combined with a multilayer perceptron and is able to automatically generate power planes across a diverse set of problems with varying levels of difficulty.…
Modibbo Adama University of Technology（莫迪博·阿达玛理工大学）
The nature of clinical data makes it difficult to quickly select, tune and apply machine learning algorithms to clinical prognosis. As a result, a lot of time is spent searching for the most appropriate machine learning algorithms applicable in clinical prognosis that contains either binary-valued or multi-valued attributes. The study set out to identify and evaluate the performance of machine learning classification schemes applied in clinical prognosis of post-operative life expectancy in the lung cancer patients. Multilayer Perceptron, J48, and the Naive Bayes algorithms were used to train …
In this paper we propose a Non-Linear Predictive Vector quantizer (PVQ) for speech coding, based on Multi-Layer Perceptrons. With this scheme we have improved the results of our previous ADPCM coder with nonlinear prediction, and we have reduced the bit rate up to 1 bit per sample.
A Multilayer Perceptron-based Fast Sunlight Assessment for the Conceptual Design of Residential Neighborhoods under Chinese Policy
Glodon Company Limited（广联达科技股份有限公司）；Tsinghua University（清华大学）；Institute for Urban Governance and Sustainable Development, Tsinghua University（清华大学城市治理与可持续发展研究院）
We empirically investigate the (negative) expected accuracy as an alternative loss function to cross entropy (negative log likelihood) for classification tasks. Coupled with softmax activation, it has small derivatives over most of its domain, and is therefore hard to optimize. A modified, leaky version is evaluated on a variety of classification tasks, including digit recognition, image classification, sequence tagging and tree tagging, using a variety of neural architectures such as logistic regression, multilayer perceptron, CNN, LSTM and Tree-LSTM. We show that it yields comparable or bett…
Providing the users of a dynamic tolling system with predictions of tolling prices and the travel time difference between the toll road and the alternative routes enables them to make their travel decisions before starting their trip. This study aims to provide accurate predictions of tolling price through training and testing random forest, multilayer perceptron, and long short-term memory models and compare them with the current situation that the best prediction is extending the current toll to the next timesteps. The prediction time horizon includes five 6-minute time intervals ahead of th…
We establish estimations for the parameters of the output distribution for the softmax activation function using the probit function. As an application, we develop a new efficient Bayesian learning algorithm for fully connected neural networks, where training and predictions are performed within the Bayesian inference framework in closed-form. This approach allows sequential learning and requires no computationally expensive gradient calculation and Monte Carlo sampling. Our work generalizes the Bayesian algorithm for a single perceptron for binary classification in \cite{H} to multi-layer per…
Spatial Gated Multi-Layer Perceptron for Land Use and Land Cover Mapping
Simon Fraser University（西蒙菲莎大学）；Alipurduar Government Engineering and Management College（阿里布尔杜阿尔政府工程与管理学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Lancaster University（兰卡斯特大学）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心）；Institute of Advanced Research in Artificial Intelligence (IARAI)（人工智能高级研究所）
Accurate aircraft-mass estimation is critical to airlines from the safety-management and performance-optimization viewpoints. Overloading an aircraft with passengers and baggage might result in a safety hazard. In contrast, not fully utilizing an aircraft's payload-carrying capacity undermines its operational efficiency and airline profitability. However, accurate determination of the aircraft mass for each operating flight is not feasible because it is impractical to weigh each aircraft component, including the payload. The existing methods for aircraft-mass estimation are dependent on the ai…
Technical University of Berlin（柏林工业大学）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）
With the growing demand for deploying deep learning models to the "edge", it is paramount to develop techniques that allow to execute state-of-the-art models within very tight and limited resource constraints. In this work we propose a software-hardware optimization paradigm for obtaining a highly efficient execution engine of deep neural networks (DNNs) that are based on fully-connected layers. Our approach is centred around compression as a means for reducing the area as well as power requirements of, concretely, multilayer perceptrons (MLPs) with high predictive performances. Firstly, we de…
Choroid plexuses (CP) are structures of the ventricles of the brain which produce most of the cerebrospinal fluid (CSF). Several postmortem and in vivo studies have pointed towards their role in the inflammatory process in multiple sclerosis (MS). Automatic segmentation of CP from MRI thus has high value for studying their characteristics in large cohorts of patients. To the best of our knowledge, the only freely available tool for CP segmentation is FreeSurfer but its accuracy for this specific structure is poor. In this paper, we propose to automatically segment CP from non-contrast enhanced…
