Fuzzy c-means based clustering algorithms are frequently used for Takagi-Sugeno-Kang (TSK) fuzzy classifier antecedent parameter estimation. One rule is initialized from each cluster. However, most of these clustering algorithms are unsupervised, which waste valuable label information in the training data. This paper proposes a supervised enhanced soft subspace clustering (SESSC) algorithm, which considers simultaneously the within-cluster compactness, between-cluster separation, and label information in clustering. It can effectively deal with high-dimensional data, be used as a classifier alone, or be integrated into a TSK fuzzy classifier to further improve its performance. Experiments on nine UCI datasets from various application domains demonstrated that SESSC based initialization outperformed other clustering approaches, especially when the number of rules is small.
NEC Laboratories Europe（NEC欧洲实验室）；Paderborn University（帕德博恩大学）
The problem of adaptive learning from evolving and possibly non-stationary data streams has attracted a lot of interest in machine learning in the recent past, and also stimulated research in related fields, such as computational intelligence and fuzzy systems. In particular, several rule-based methods for the incremental induction of regression models have been proposed. In this paper, we develop a method that combines the strengths of two existing approaches rooted in different learning paradigms. More concretely, our method adopts basic principles of the state-of-the-art learning algorithm …
Huazhong University of Science and Technology（华中科技大学）
Takagi-Sugeno-Kang (TSK) fuzzy systems are flexible and interpretable machine learning models; however, they may not be easily optimized when the data size is large, and/or the data dimensionality is high. This paper proposes a mini-batch gradient descent (MBGD) based algorithm to efficiently and effectively train TSK fuzzy classifiers. It integrates two novel techniques: 1) uniform regularization (UR), which forces the rules to have similar average contributions to the output, and hence to increase the generalization performance of the TSK classifier; and, 2) batch normalization (BN), which e…
Huazhong University of Science and Technology（华中科技大学）
This paper presents PyTSK, a Python toolbox for developing Takagi-Sugeno-Kang (TSK) fuzzy systems. Based on scikit-learn and PyTorch, PyTSK allows users to optimize TSK fuzzy systems using fuzzy clustering or mini-batch gradient descent (MBGD) based algorithms. Several state-of-the-art MBGD-based optimization algorithms are implemented in the toolbox, which can improve the generalization performance of TSK fuzzy systems, especially for big data applications. PyTSK can also be easily extended and customized for more complicated algorithms, such as modifying the structure of TSK fuzzy systems, d…
Huazhong University of Science and Technology（华中科技大学）；School of Artificial Intelligence and Automation（人工智能与自动化学院）；Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control（教育部图像处理与智能控制重点实验室）
To effectively train Takagi-Sugeno-Kang (TSK) fuzzy systems for regression problems, a Mini-Batch Gradient Descent with Regularization, DropRule, and AdaBound (MBGD-RDA) algorithm was recently proposed. It has demonstrated superior performances; however, there are also some limitations, e.g., it does not allow the user to specify the number of rules directly, and only Gaussian MFs can be used. This paper proposes two variants of MBGD-RDA to remedy these limitations, and show that they outperform the original MBGD-RDA and the classical ANFIS algorithms with the same number of rules. Furthermore…
Huzhou University（湖州师范学院）；Zhejiang Normal University（浙江师范大学）；Jiangnan University（江南大学）；Huazhong University of Science and Technology（华中科技大学）；Hangzhou Dianzi University（杭州电子科技大学）；Zhejiang University（浙江大学）
High-order Takagi-Sugeno-Kang (TSK) fuzzy classifiers possess powerful classification performance yet have fewer fuzzy rules, but always be impaired by its exponential growth training time and poorer interpretability owing to High-order polynomial used in consequent part of fuzzy rule, while Low-order TSK fuzzy classifiers run quickly with high interpretability, however they usually require more fuzzy rules and perform relatively not very well. Address this issue, a novel TSK fuzzy classifier embeded with knowledge distillation in deep learning called HTSK-LLM-DKD is proposed in this study. HT…
Huazhong University of Science and Technology（华中科技大学）
Takagi-Sugeno-Kang (TSK) fuzzy systems are very useful machine learning models for regression problems. However, to our knowledge, there has not existed an efficient and effective training algorithm that ensures their generalization performance, and also enables them to deal with big data. Inspired by the connections between TSK fuzzy systems and neural networks, we extend three powerful neural network optimization techniques, i.e., mini-batch gradient descent, regularization, and AdaBound, to TSK fuzzy systems, and also propose three novel techniques (DropRule, DropMF, and DropMembership) spe…
Jiangnan University（江南大学）；The Hong Kong Polytechnic University（香港理工大学）
The superior interpretability and uncertainty modeling ability of Takagi-Sugeno-Kang fuzzy system (TSK FS) make it possible to describe complex nonlinear systems intuitively and efficiently. However, classical TSK FS usually adopts the whole feature space of the data for model construction, which can result in lengthy rules for high-dimensional data and lead to degeneration in interpretability. Furthermore, for highly nonlinear modeling task, it is usually necessary to use a large number of rules which further weakens the clarity and interpretability of TSK FS. To address these issues, a conci…
Huazhong University of Science and Technology（华中科技大学）
Takagi-Sugeno-Kang (TSK) fuzzy system with Gaussian membership functions (MFs) is one of the most widely used fuzzy systems in machine learning. However, it usually has difficulty handling high-dimensional datasets. This paper explores why TSK fuzzy systems with Gaussian MFs may fail on high-dimensional inputs. After transforming defuzzification to an equivalent form of softmax function, we find that the poor performance is due to the saturation of softmax. We show that two defuzzification operations, LogTSK and HTSK, the latter of which is first proposed in this paper, can avoid the saturatio…
Huazhong University of Science and Technology（华中科技大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
To effectively optimize Takagi-Sugeno-Kang (TSK) fuzzy systems for regression problems, a mini-batch gradient descent with regularization, DropRule, and AdaBound (MBGD-RDA) algorithm was recently proposed. This paper further proposes FCM-RDpA, which improves MBGD-RDA by replacing the grid partition approach in rule initialization by fuzzy c-means clustering, and AdaBound by Powerball AdaBelief, which integrates recently proposed Powerball gradient and AdaBelief to further expedite and stabilize parameter optimization. Extensive experiments on 22 regression datasets with various sizes and dimen…
School of Wangzheng Microelectronics, Changzhou University（常州大学王正微电子学院）；School of Electrical Engineering and Automation, Changshu Institute of Technology（常熟理工学院电气与自动化工程学院）
Huazhong University of Science and Technology（华中科技大学）；University of Technology, Sydney（悉尼科技大学）
Fuzzy systems have achieved great success in numerous applications. However, there are still many challenges in designing an optimal fuzzy system, e.g., how to efficiently optimize its parameters, how to balance the trade-off between cooperations and competitions among the rules, how to overcome the curse of dimensionality, how to increase its generalization ability, etc. Literature has shown that by making appropriate connections between fuzzy systems and other machine learning approaches, good practices from other domains may be used to improve the fuzzy systems, and vice versa. This paper g…
Transfer learning can address the learning tasks of unlabeled data in the target domain by leveraging plenty of labeled data from a different but related source domain. A core issue in transfer learning is to learn a shared feature space in where the distributions of the data from two domains are matched. This learning process can be named as transfer representation learning (TRL). The feature transformation methods are crucial to ensure the success of TRL. The most commonly used feature transformation method in TRL is kernel-based nonlinear mapping to the high-dimensional space followed by li…
Centro de Investigación en Tecnoloxías da Información (CITIUS)（信息技术研究中心（CITIUS））；Universidade de Santiago de Compostela（圣地亚哥德孔波斯特拉大学）
In regression problems, the use of TSK fuzzy systems is widely extended due to the precision of the obtained models. Moreover, the use of simple linear TSK models is a good choice in many real problems due to the easy understanding of the relationship between the output and input variables. In this paper we present FRULER, a new genetic fuzzy system for automatically learning accurate and simple linguistic TSK fuzzy rule bases for regression problems. In order to reduce the complexity of the learned models while keeping a high accuracy, the algorithm consists of three stages: instance selectio…
A Novel TSK Fuzzy System Incorporating Multi-view Collaborative Transfer Learning for Personalized Epileptic EEG Detection
School of Artificial Intelligence and Computer Science, Jiangnan University and Engineering Research Center of Intelligent Technology for Healthcare, Ministry of Education（江南大学人工智能与计算机科学学院，智能医疗技术工程研究中心，教育部）；School of Computer and Information Engineering, Institute for Artificial Intelligence, Shanghai Polytechnic University（上海理工大学计算机与信息工程学院，人工智能研究院）；School of Internet of Things Engineering, Wuxi University（无锡大学物联网工程学院）
On-board Sonar Data Classification for Path Following in Underwater Vehicles using Fast Interval Type-2 Fuzzy Extreme Learning Machine
Hamlyn Centre for Robotic Surgery, Imperial College London（机器人手术哈明中心，帝国理工学院伦敦分校）；Center for Engineering and Industrial Development (CIDESI)（工程与工业发展中心（CIDESI））
The University of Suwon（水原大学）；Linyi University（临沂大学）；University of Alberta（阿尔伯塔大学）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）；Beijing Normal University（北京师范大学）
In the conventional Takagi-Sugeno-Kang (TSK)-type fuzzy models, constant or linear functions are usually utilized as the consequent parts of the fuzzy rules, but they cannot effectively describe the behavior within local regions defined by the antecedent parts. In this article, a theoretical and practical design methodology is developed to address this problem. First, the information granulation (Fuzzy C-Means) method is applied to capture the structure in the data and split the input space into subspaces, as well as form the antecedent parts. Second, the quadratic polynomials (QPs) are employ…
Data mining techniques can be used to discover useful patterns by exploring and analyzing data and it's feasible to synergitically combine machine learning tools to discover fuzzy classification rules.In this paper, an adaptive Neuro fuzzy network with TSK fuzzy type and an improved quantum subtractive clustering has been developed. Quantum clustering (QC) is an intuition from quantum mechanics which uses Schrodinger potential and time-consuming gradient descent method. The principle advantage and shortcoming of QC is analyzed and based on its shortcomings, an improved algorithm through a subt…
