Predictive clustering trees (PCTs) are a well established generalization of standard decision trees, which can be used to solve a variety of predictive modeling tasks, including structured output prediction. Combining them into ensembles yields state-of-the-art performance. Furthermore, the ensembles of PCTs can be interpreted by calculating feature importance scores from the learned models. However, their learning time scales poorly with the dimensionality of the output space. This is often problematic, especially in (hierarchical) multi-label classification, where the output can consist of hundreds of potential labels. Also, learning of PCTs can not exploit the sparsity of data to improve the computational efficiency, which is common in both input (molecular fingerprints, bag of words representations) and output spaces (in multi-label classification, examples are often labeled with only a fraction of possible labels). In this paper, we propose oblique predictive clustering trees, capable of addressing these limitations. We design and implement two methods for learning oblique splits that contain linear combinations of features in the tests, hence a split corresponds to an arbitrary hyperplane in the input space. The methods are efficient for high dimensional data and capable of exploiting sparse data. We experimentally evaluate the proposed methods on 60 benchmark datasets for 6 predictive modeling tasks. The results of the experiments show that oblique predictive clustering trees achieve performance on-par with state-of-the-art methods and are orders of magnitude faster than standard PCTs. We also show that meaningful feature importance scores can be extracted from the models learned with the proposed methods.
Faculty of Nursing and Physiotherapy, University of Salamanca（萨拉曼卡大学护理与物理治疗学院）；University Hospital of Salamanca（萨拉曼卡大学医院）；Department of Computing and Automation, University of Salamanca（萨拉曼卡大学计算与自动化系）；Department of Statistics, University of Salamanca（萨拉曼卡大学统计系）
Early detection of patients vulnerable to infections acquired in the hospital environment is a challenge in current health systems given the impact that such infections have on patient mortality and healthcare costs. This work is focused on both the identification of risk factors and the prediction of healthcare-associated infections in intensive-care units by means of machine-learning methods. The aim is to support decision making addressed at reducing the incidence rate of infections. In this field, it is necessary to deal with the problem of building reliable classifiers from imbalanced dat…
Uncertainty-Aware Extrapolation in Bayesian Oblique Trees
Jožef Stefan Institute（乔泽夫·斯蒂芬研究所）；Jožef Stefan International Postgraduate School（乔泽夫·斯蒂芬国际研究生学校）；Faculty of Information Studies in Novo mesto（诺沃梅托信息研究学院）
In the past few years co-clustering has emerged as an important data mining tool for two way data analysis. Co-clustering is more advantageous over traditional one dimensional clustering in many ways such as, ability to find highly correlated sub-groups of rows and columns. However, one of the overlooked benefits of co-clustering is that, it can be used to extract meaningful knowledge for various other knowledge extraction purposes. For example, building predictive models with high dimensional data and heterogeneous population is a non-trivial task. Co-clusters extracted from such data, which …
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of Cambridge（剑桥大学）；Alan Turing Institute（艾伦·图灵研究所）
Due to the wider availability of modern electronic health records, patient care data is often being stored in the form of time-series. Clustering such time-series data is crucial for patient phenotyping, anticipating patients' prognoses by identifying "similar" patients, and designing treatment guidelines that are tailored to homogeneous patient subgroups. In this paper, we develop a deep learning approach for clustering time-series data, where each cluster comprises patients who share similar future outcomes of interest (e.g., adverse events, the onset of comorbidities). To encourage each clu…
We propose that the Continual Learning desiderata can be achieved through a neuro-inspired architecture, grounded on Mountcastle's cortical column hypothesis. The proposed architecture involves a single module, called Self-Taught Associative Memory (STAM), which models the function of a cortical column. STAMs are repeated in multi-level hierarchies involving feedforward, lateral and feedback connections. STAM networks learn in an unsupervised manner, based on a combination of online clustering and hierarchical predictive coding. This short paper only presents the architecture and its connectio…
Computational Imaging Research Lab (CIR)（计算成像研究实验室（CIR））；Department of Biomedical Imaging and Image-guided Therapy（生物医学成像与影像引导治疗系）；Medical University of Vienna（维也纳医科大学）
Predictive marker patterns in imaging data are a means to quantify disease and progression, but their identification is challenging, if the underlying biology is poorly understood. Here, we present a method to identify predictive texture patterns in medical images in an unsupervised way. Based on deep clustering networks, we simultaneously encode and cluster medical image patches in a low-dimensional latent space. The resulting clusters serve as features for disease staging, linking them to the underlying disease. We evaluate the method on 70 T1-weighted magnetic resonance images of patients w…
Guise AI, Inc.（Guise AI 公司）；Applied Computational Intelligence Laboratory, Missouri University of Science and Technology（密苏里科技大学应用计算智能实验室）
This paper presents an adaptive resonance theory predictive mapping (ARTMAP) model which uses incremental cluster validity indices (iCVIs) to perform unsupervised learning, namely iCVI-ARTMAP. Incorporating iCVIs to the decision-making and many-to-one mapping capabilities of ARTMAP can improve the choices of clusters to which samples are incrementally assigned. These improvements are accomplished by intelligently performing the operations of swapping sample assignments between clusters, splitting and merging clusters, and caching the values of variables when iCVI values need to be recomputed. …
Predictive Clustering of Vessel Behavior Based on Hierarchical Trajectory Representation
School of Computer Science and Artificial Intelligence, Wuhan University of Technology（武汉理工大学计算机科学与人工智能学院）；College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与电子工程学院）；School of Navigation, Wuhan University of Technology（武汉理工大学航海学院）
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Carney Institute for Brain Science（卡尼脑科学研究所）；Brown University（布朗大学）
Recent advances in reinforcement-learning research have demonstrated impressive results in building algorithms that can out-perform humans in complex tasks. Nevertheless, creating reinforcement-learning systems that can build abstractions of their experience to accelerate learning in new contexts still remains an active area of research. Previous work showed that reward-predictive state abstractions fulfill this goal, but have only be applied to tabular settings. Here, we provide a clustering algorithm that enables the application of such state abstractions to deep learning settings, providing…
Time series clustering based on prediction accuracy of global forecasting models
Research Center for Information and Communication Technologies (CITIC)（信息与通信技术研究中心（CITIC））；University of A Coruña（拉科鲁尼亚大学）；The University of Sydney Business School（悉尼大学商学院）
Orange Labs（Orange实验室）；AgroParisTech, Université Paris-Saclay（巴黎高科农业学院，巴黎萨克雷大学）；LIM-EA2525, Université de La Réunion（留尼汪大学LIM-EA2525实验室）
Supervised classification can be effective for prediction but sometimes weak on interpretability or explainability (XAI). Clustering, on the other hand, tends to isolate categories or profiles that can be meaningful but there is no guarantee that they are useful for labels prediction. Predictive clustering seeks to obtain the best of the two worlds. Starting from labeled data, it looks for clusters that are as pure as possible with regards to the class labels. One technique consists in tweaking a clustering algorithm so that data points sharing the same label tend to aggregate together. With d…
Multilayer GNN for Predictive Maintenance and Clustering in Power Grids
organization= Industrial \& Manufacturing Engineering, North Dakota State University , addressline= NDSU Dept. 2485, PO Box 6050 , city= Fargo , postcode= 58108-6050 , state= ND , country= USA；organization= Department of Engineering Technology \& Construction Management, University of North Carolina at Charlotte , postcode= 28223 , state= NC , country= USA；organization= Department of Industrial \& Systems Engineering, North Carolina Agriculture \& Tech State University , city= Greensboro , postcode= 27411 , state= NC , country= USA；organization= Department of Industrial \& Management Systems Engineering, University of South Florida , addressline= 4202 East Fowler Avenue, ENG 030 , city= Tampa , postcode= 33620 , state= FL , country= USA
We formally prove the connection between k-means clustering and the predictions of neural networks based on the softmax activation layer. In existing work, this connection has been analyzed empirically, but it has never before been mathematically derived. The softmax function partitions the transformed input space into cones, each of which encompasses a class. This is equivalent to putting a number of centroids in this transformed space at equal distance from the origin, and k-means clustering the data points by proximity to these centroids. Softmax only cares in which cone a data point falls,…
Neural networks are capable of learning rich, nonlinear feature representations shown to be beneficial in many predictive tasks. In this work, we use these models to explore the use of geographical features in predicting colorectal cancer survival curves for patients in the state of Iowa, spanning the years 1989 to 2012. Specifically, we compare model performance using a newly defined metric -- area between the curves (ABC) -- to assess (a) whether survival curves can be reasonably predicted for colorectal cancer patients in the state of Iowa, (b) whether geographical features improve predicti…
Semi-supervised Clustering Through Representation Learning of Large-scale EHR Data
Department of Biostatistics, Harvard T.H. Chan School of Public Health（哈佛T.H. 洪学校公共卫生学院生物统计学系）；Department of Mathematical Sciences, Bentley University（贝纳特大学数学科学系）；Department of Neurology, University of Pittsburgh（匹兹堡大学神经病学系）；Department of Biostatistics, Peking University Health Science Center（北京大学健康科学中心生物统计学系）；Beijing International Center for Mathematical Research, Peking University（北京大学北京国际数学研究中心）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）
In the era of big data, we are often facing the challenge of data heterogeneity and the lack of label information simultaneously. In the financial domain (e.g., fraud detection), the heterogeneous data may include not only numerical data (e.g., total debt and yearly income), but also text and images (e.g., financial statement and invoice images). At the same time, the label information (e.g., fraud transactions) may be missing for building predictive models. To address these challenges, many state-of-the-art multi-view clustering methods have been proposed and achieved outstanding performance.…
