In tis paper we consider approaches for time series forecasting based on deep neural networks and neuro-fuzzy nets. Also, we make short review of researches in forecasting based on various models of ANFIS models. Deep Learning has proven to be an effective method for making highly accurate predictions from complex data sources. Also, we propose our models of DL and Neuro-Fuzzy Networks for this task. Finally, we show possibility of using these models for data science tasks. This paper presents also an overview of approaches for incorporating rule-based methodology into deep learning neural networks.
China University of Petroleum (East China)（中国石油大学（华东））；Indian Statistical Institute（印度统计研究所）
A major limitation of fuzzy or neuro-fuzzy systems is their failure to deal with high-dimensional datasets. This happens primarily due to the use of T-norm, particularly, product or minimum (or a softer version of it). Thus, there are hardly any work dealing with datasets with dimensions more than hundred or so. Here, we propose a neuro-fuzzy framework that can handle datasets with dimensions even more than 7000! In this context, we propose an adaptive softmin (Ada-softmin) which effectively overcomes the drawbacks of ``numeric underflow" and ``fake minimum" that arise for existing fuzzy syste…
Several adaptation techniques have been investigated to optimize fuzzy inference systems. Neural network learning algorithms have been used to determine the parameters of fuzzy inference system. Such models are often called as integrated neuro-fuzzy models. In an integrated neuro-fuzzy model there is no guarantee that the neural network learning algorithm converges and the tuning of fuzzy inference system will be successful. Success of evolutionary search procedures for optimization of fuzzy inference system is well proven and established in many application areas. In this paper, we will explo…
Nanyang Technological University（南洋理工大学）；University of Alberta（阿尔伯塔大学）；Monash University（莫纳什大学）
Existing FNNs are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This paper proposes a novel self-organizing deep FNN, namely DEVFNN. Fuzzy rules can be automatically extracted from data streams or removed if they play limited role during their lifespan. The structure of the network can be deepened on demand by stacking additional layers using a drift detection method which not only detects the covariate drift, variations of input space, but also accurately identifies the real drift, dynamic changes of both feature space …
Technical Engineering College / Northern Technical University（北方技术大学技术工程学院）
This paper describes the design and development of a prototype technique for artificial intelligence based on the fusion of genetic algorithm, neural network and fuzzy logic. It starts by establishing a relationship between the neural network and fuzzy logic. Then, it combines the genetic algorithm with them. Information fusions are at the confidence level, where matching scores can be reported and discussed. The technique is called the Genetic Neuro-Fuzzy (GNF). It can be used for high accuracy real-time environments.
Departamento de Matemática（数学系）；Instituto Superior de Engenharia de Lisboa（里斯本高等工程学院）
This work describes a methodology to combine logic-based systems and connectionist systems. Our approach uses finite truth valued Łukasiewicz logic, where we take advantage of fact what in this type of logics every connective can be define by a neuron in an artificial network having by activation function the identity truncated to zero and one. This allowed the injection of first-order formulas in a network architecture, and also simplified symbolic rule extraction. Our method trains a neural network using Levenderg-Marquardt algorithm, where we restrict the knowledge dissemination in the netw…
Interpretable Dual-Filter Fuzzy Neural Networks for Affective Brain-Computer Interfaces
University of Technology Sydney（悉尼科技大学）；Australian AI Institute（澳大利亚人工智能研究所）；Henan University（河南大学）；Indian Statistical Institute（印度统计研究所）；Chinese Academy of Sciences（中国科学院）
In this paper a novel neuro-fuzzy system is proposed where its learning is based on the creation of fuzzy relations by using new implication method without utilizing any exact mathematical techniques. Then, a simple memristor crossbar-based analog circuit is designed to implement this neuro-fuzzy system which offers very interesting properties. In addition to high connectivity between neurons and being fault-tolerant, all synaptic weights in our proposed method are always non-negative and there is no need to precisely adjust them. Finally, this structure is hierarchically expandable and can co…
Kharkiv National University of Radio Electronics（哈尔科夫国立无线电电子大学）
An architecture of a new neuro-fuzzy system is proposed. The basic idea of this approach is to tune both synaptic weights and membership functions with the help of the supervised learning and self-learning paradigms. The approach to solving the problem has to do with evolving online neuro-fuzzy systems that can process data under uncertainty conditions. The results prove the effectiveness of the developed architecture and the learning procedure.
Xsensor Technology Corp（Xsensor科技公司）；University of Alberta（阿尔伯塔大学）
The combination of neural network and fuzzy systems into neuro-fuzzy systems integrates fuzzy reasoning rules into the connectionist networks. However, the existing neuro-fuzzy systems are developed under shallow structures having lower generalization capacity. We propose the first end-to-end deep neuro-fuzzy network and investigate its application for image classification. Two new operations are developed based on definitions of Takagi-Sugeno-Kang (TSK) fuzzy model namely fuzzy inference operation and fuzzy pooling operations; stacks of these operations comprise the layers in this network. We…
Central China Normal University（华中师范大学）；Kharkiv National University of Radio Electronics（哈尔科夫国立无线电电子大学）
An evolving weighted neuro-neo-fuzzy-ANARX model and its learning procedures are introduced in the article. This system is basically used for time series forecasting. This system may be considered as a pool of elements that process data in a parallel manner. The proposed evolving system may provide online processing data streams.
Universidad Santiago de Compostela（圣地亚哥-德孔波斯特拉大学）；Universidad Autónoma de Madrid（马德里自治大学）；Universidad Pontificia de Comillas（科米利亚斯主教大学）
Imagery is frequently used to model, represent and communicate knowledge. In particular, graphs are one of the most powerful tools, being able to represent relations between objects. Causal relations are frequently represented by directed graphs, with nodes denoting causes and links denoting causal influence. A causal graph is a skeletal picture, showing causal associations and impact between entities. Common methods used for graphically representing causal scenarios are neurons, truth tables, causal Bayesian networks, cognitive maps and Petri Nets. Causality is often defined in terms of prece…
Kharkiv National University of Radio Electronics（哈尔科夫国立无线电电子大学）
A new architecture and learning algorithms for the multidimensional hybrid cascade neural network with neuron pool optimization in each cascade are proposed in this paper. The proposed system differs from the well-known cascade systems in its capability to process multidimensional time series in an online mode, which makes it possible to process non-stationary stochastic and chaotic signals with the required accuracy. Compared to conventional analogs, the proposed system provides computational simplicity and possesses both tracking and filtering capabilities.
Sharif University of Technology（谢里夫理工大学）；University of Zanjan（赞詹大学）
In this paper, first we present a new explanation for the relation between logical circuits and artificial neural networks, logical circuits and fuzzy logic, and artificial neural networks and fuzzy inference systems. Then, based on these results, we propose a new neuro-fuzzy computing system which can effectively be implemented on the memristor-crossbar structure. One important feature of the proposed system is that its hardware can directly be trained using the Hebbian learning rule and without the need to any optimization. The system also has a very good capability to deal with huge number …
Huawei International（华为国际）；College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）
With the development of deep learning processors and accelerators, deep learning models have been widely deployed on edge devices as part of the Internet of Things. Edge device models are generally considered as valuable intellectual properties that are worth for careful protection. Unfortunately, these models have a great risk of being stolen or illegally copied. The existing model protections using encryption algorithms are suffered from high computation overhead which is not practical due to the limited computing capacity on edge devices. In this work, we propose a light-weight, practical, …
Over past several years, deep learning has achieved huge successes in various applications. However, such a data-driven approach is often criticized for lack of interpretability. Recently, we proposed artificial quadratic neural networks consisting of second-order neurons in potentially many layers. In each second-order neuron, a quadratic function is used in the place of the inner product in a traditional neuron, and then undergoes a nonlinear activation. With a single second-order neuron, any fuzzy logic operation, such as XOR, can be implemented. In this sense, any deep network constructed …
This article presents the Neo-Fuzzy-Neuron Modified by Kohonen Network (NFN-MK), an hybrid computational model that combines fuzzy system technique and artificial neural networks. Its main task consists in the automatic generation of membership functions, in particular, triangle forms, aiming a dynamic modeling of a system. The model is tested by simulating real systems, here represented by a nonlinear mathematical function. Comparison with the results obtained by traditional neural networks, and correlated studies of neurofuzzy systems applied in system identification area, shows that the NFN…
A Neural Network, in general, is not considered to be a good solver of mathematical and binary arithmetic problems. However, networks have been developed for such problems as the XOR circuit. This paper presents a technique for the implementation of the Half-adder circuit using the CoActive Neuro-Fuzzy Inference System (CANFIS) Model and attempts to solve the problem using the NeuroSolutions 5 Simulator. The paper gives the experimental results along with the interpretations and possible applications of the technique.
Yazd University（亚兹德大学）；Islamic Azad University of Yazd（亚兹德伊斯兰阿扎德大学）；Islamic Azad University of Meybod（梅博德伊斯兰阿扎德大学）
In this study, a hybrid method based on an Adaptive Neuro-Fuzzy Inference System (ANFIS) and Particle Swarm Optimization (PSO) for diagnosing Liver disorders (ANFIS-PSO) is introduced. This smart diagnosis method deals with a combination of making an inference system and optimization process which tries to tune the hyper-parameters of ANFIS based on the data-set. The Liver diseases characteristics are taken from the UCI Repository of Machine Learning Databases. The number of these characteristic attributes are 7, and the sample number is 354. The right diagnosis performance of the ANFIS-PSO in…
Gradient-Based Neuroplastic Adaptation for Concurrent Optimization of Neuro-Fuzzy Networks
Department of Analytics and Information Systems at Western Kentucky University（威斯康星大学西部分校分析与信息系统系）；Department of Computer Science at North Carolina State University（北卡罗来纳州立大学计算机科学系）
In this paper, we are introducing a novel model of artificial intelligence, the functional neural network for modeling of human decision-making processes. This neural network is composed of multiple artificial neurons racing in the network. Each of these neurons has a similar structure programmed independently by the users and composed of an intention wheel, a motor core and a sensory core representing the user itself and racing at a specific velocity. The mathematics of the neuron's formulation and the racing mechanism of multiple nodes in the network will be discussed, and the group decision…
University of Technology Sydney（悉尼科技大学）；Advanced Analytics Institute（高级分析研究所）；Faculty of Engineering and Information Technology（工程与信息技术学院）
General fuzzy min-max neural network (GFMMNN) is one of the efficient neuro-fuzzy systems for data classification. However, one of the downsides of its original learning algorithms is the inability to handle and learn from the mixed-attribute data. While categorical features encoding methods can be used with the GFMMNN learning algorithms, they exhibit a lot of shortcomings. Other approaches proposed in the literature are not suitable for on-line learning as they require entire training data available in the learning phase. With the rapid change in the volume and velocity of streaming data in …
In this article, we propose a combination of an noise-reduction algorithm based on Singular Spectrum Analysis (SSA) and a standard feedforward neural prediction model. Basically, the proposed algorithm consists of two different steps: data preprocessing based on the SSA filtering method and step-by-step training procedure in which we use a simple feedforward multilayer neural network with backpropagation learning. The proposed noise-reduction procedure successfully removes most of the noise. That increases long-term predictability of the processed dataset comparison with the raw dataset. The m…
Recurrent Neural Networks (RNNs) are a key technology for applications such as automatic speech recognition or machine translation. Unlike conventional feed-forward DNNs, RNNs remember past information to improve the accuracy of future predictions and, therefore, they are very effective for sequence processing problems. For each application run, recurrent layers are executed many times for processing a potentially large sequence of inputs (words, images, audio frames, etc.). In this paper, we observe that the output of a neuron exhibits small changes in consecutive invocations.~We exploit this…
The U.S. water distribution system contains thousands of miles of pipes constructed from different materials, and of various sizes, and age. These pipes suffer from physical, environmental, structural and operational stresses, causing deterioration which eventually leads to their failure. Pipe deterioration results in increased break rates, reduced hydraulic capacity, and detrimental impacts on water quality. Therefore, it is crucial to use accurate models to forecast deterioration rates along with estimating the remaining useful life of the pipes to implement essential interference plans in o…
This paper describes application of information granulation theory, on the analysis of "lugeon data". In this manner, using a combining of Self Organizing Map (SOM) and Neuro-Fuzzy Inference System (NFIS), crisp and fuzzy granules are obtained. Balancing of crisp granules and sub- fuzzy granules, within non fuzzy information (initial granulation), is rendered in open-close iteration. Using two criteria, "simplicity of rules "and "suitable adaptive threshold error level", stability of algorithm is guaranteed. In other part of paper, rough set theory (RST), to approximate analysis, has been empl…
Federal University of Minas Gerais（米纳斯吉拉斯联邦大学）；Federal Institute of Education Science and Technology of Northern Minas Gerais（北米纳斯吉拉斯联邦教育科学与技术学院）
Among various soft computing approaches for time series forecasting, Fuzzy Cognitive Maps (FCM) have shown remarkable results as a tool to model and analyze the dynamics of complex systems. FCM have similarities to recurrent neural networks and can be classified as a neuro-fuzzy method. In other words, FCMs are a mixture of fuzzy logic, neural network, and expert system aspects, which act as a powerful tool for simulating and studying the dynamic behavior of complex systems. The most interesting features are knowledge interpretability, dynamic characteristics and learning capability. The goal …
Amirkabir University of Technology（阿米尔卡比尔理工大学）；University of Alberta（阿尔伯塔大学）
This paper pursues some applications of Rough Set Theory (RST) and neural-fuzzy model to analysis of "lugeon data". In the manner, using Self Organizing Map (SOM) as a pre-processing the data are scaled and then the dominant rules by RST, are elicited. Based on these rules variations of permeability in the different levels of Shivashan dam, Iran has been highlighted. Then, via using a combining of SOM and an adaptive Neuro-Fuzzy Inference System (NFIS) another analysis on the data was carried out. Finally, a brief comparison between the obtained results of RST and SOM-NFIS (briefly SONFIS) has…
Solar activity It is vital to accurately predict solar activity, in order to decrease the plausible damage of electronic equipment in the event of a large high-intensity solar eruption. Recently, we have proposed BELFIS (Brain Emotional Learning-based Fuzzy Inference System) as a tool for the forecasting of chaotic systems. The structure of BELFIS is designed based on the neural structure of fear conditioning. The function of BELFIS is implemented by assigning adaptive networks to the components of the BELFIS structure. This paper especially focuses on performance evaluation of BELFIS as a pre…
Technical University of Catalonia（加泰罗尼亚理工大学）；University of the Basque Country（巴斯克大学）
This paper develops a two-layer neural network in which the neuron model computes a user-defined similarity function between inputs and weights. The neuron transfer function is formed by composition of an adapted logistic function with the mean of the partial input-weight similarities. The resulting neuron model is capable of dealing directly with variables of potentially different nature (continuous, fuzzy, ordinal, categorical). There is also provision for missing values. The network is trained using a two-stage procedure very similar to that used to train a radial basis function (RBF) neura…
