In many scientific tasks we are interested in discovering whether there exist any correlations in our data. This raises many questions, such as how to reliably and interpretably measure correlation between a multivariate set of attributes, how to do so without having to make assumptions on distribution of the data or the type of correlation, and, how to efficiently discover the top-most reliably correlated attribute sets from data. In this paper we answer these questions for discovery tasks in categorical data.
In particular, we propose a corrected-for-chance, consistent, and efficient estimator for normalized total correlation, by which we obtain a reliable, naturally interpretable, non-parametric measure for correlation over multivariate sets. For the discovery of the top-k correlated sets, we derive an effective algorithmic framework based on a tight bounding function. This framework offers exact, approximate, and heuristic search. Empirical evaluation shows that already for small sample sizes the estimator leads to low-regret optimization outcomes, while the algorithms are shown to be highly effective for both large and high-dimensional data. Through two case studies we confirm that our discovery framework identifies interesting and meaningful correlations.
With the growing size of data sets, feature selection becomes increasingly important. Taking interactions of original features into consideration will lead to extremely high dimension, especially when the features are categorical and one-hot encoding is applied. This makes it more worthwhile mining useful features as well as their interactions. Association rule mining aims to extract interesting correlations between items, but it is difficult to use rules as a qualified classifier themselves. Drawing inspiration from association rule mining, we come up with a method that uses association rules…
Dortmund University of Applied Sciences and Arts（多特蒙德应用科学与艺术大学）
We introduce a method that uses low-rank approximations of cross-correlation matrices in mixed continuous and categorical Gaussian Process models. This new method -- called Low-Rank Correlation (LRC) -- offers the ability to flexibly adapt the number of parameters to the problem at hand by choosing an appropriate rank of the approximation. Furthermore, we present a systematic approach of defining test functions that can be used for assessing the accuracy of models or optimization methods that are concerned with both continuous and categorical inputs. We compare LRC to existing approaches of mo…
Information Sciences Institute（信息科学研究所）；University of Southern California（南加州大学）
We introduce a method to learn a hierarchy of successively more abstract representations of complex data based on optimizing an information-theoretic objective. Intuitively, the optimization searches for a set of latent factors that best explain the correlations in the data as measured by multivariate mutual information. The method is unsupervised, requires no model assumptions, and scales linearly with the number of variables which makes it an attractive approach for very high dimensional systems. We demonstrate that Correlation Explanation (CorEx) automatically discovers meaningful structure…
The challenge of solving data mining problems in e-commerce applications such as recommendation system (RS) and click-through rate (CTR) prediction is how to make inferences by constructing combinatorial features from a large number of categorical features while preserving the interpretability of the method. In this paper, we propose Automatic Embedded Feature Engineering(AEFE), an automatic feature engineering framework for representing categorical features, which consists of various components including custom paradigm feature construction and multiple feature selection. By selecting the pot…
Business intelligence (BI) is any knowledge derived from existing data that may be strategically applied within a business. Data mining is a technique or method for extracting BI from data using statistical data modeling. Finding relationships or correlations between the various data items that have been collected can be used to boost business performance or at the very least better comprehend what is going on. Root cause analysis (RCA) is discovering the root causes of problems or events to identify appropriate solutions. RCA can show why an event occurred and this can help in avoiding occurr…
Discovering Spatial Correlations of Earth Observations for weather forecasting by using Graph Structure Learning
Data Assimilation Group\ Institute of Atmospheric Prediction Systems 35, 5-gil, Boramae-ro Seoul Republic of Korea 07071；Network Science Lab\ Catholic University of Korea 43 Jibong-ro Bucheon Republic of Korea 14662；Data Assimilation Group\ Institute of Atmospheric Prediction Systems；Network Science Lab\ Catholic University of Korea
We consider analysis of relational data (a matrix), in which the rows correspond to subjects (e.g., people) and the columns correspond to attributes. The elements of the matrix may be a mix of real and categorical. Each subject and attribute is characterized by a latent binary feature vector, and an inferred matrix maps each row-column pair of binary feature vectors to an observed matrix element. The latent binary features of the rows are modeled via a multivariate Gaussian distribution with low-rank covariance matrix, and the Gaussian random variables are mapped to latent binary features via …
University of Cambridge（剑桥大学）；The Alan Turing Institute（阿兰·图灵研究所）；Chung-Ang University（中央大学）
Clustering time-series data in healthcare is crucial for clinical phenotyping to understand patients' disease progression patterns and to design treatment guidelines tailored to homogeneous patient subgroups. While rich temporal dynamics enable the discovery of potential clusters beyond static correlations, two major challenges remain outstanding: i) discovery of predictive patterns from many potential temporal correlations in the multi-variate time-series data and ii) association of individual temporal patterns to the target label distribution that best characterizes the underlying clinical p…
Microsoft Research（微软研究院）；University of Wisconsin - Milwaukee（威斯康星大学密尔沃基分校）
Computational methods for discovering patterns of local correlations in sequences are important in computational biology. Here we show how to determine the optimal partitioning of aligned sequences into non-overlapping segments such that positions in the same segment are strongly correlated while positions in different segments are not. Our approach involves discovering the hidden variables of a Bayesian network that interact with observed sequences so as to form a set of independent mixture models. We introduce a dynamic program to efficiently discover the optimal segmentation, or equivalentl…
Discovering and Explaining the Non-Causality of Deep Learning in SAR ATR
National University of Defense Technology（国防科技大学）；College of Electronic Science and Technology, National University of Defense Technology（国防科技大学电子科学学院）
Inferring causal directions on discrete and categorical data is an important yet challenging problem. Even though the additive noise models (ANMs) approach can be adapted to the discrete data, the functional structure assumptions make it not applicable on categorical data. Inspired by the principle that the cause and mechanism are independent, various methods have been developed, leveraging independence tests such as the distance correlation measure. In this work, we take an alternative perspective and propose a subsampling-based method to test the independence between the generating schemes o…
Discovering Latent Causal Graphs from Spatiotemporal Data
Department of Computer Science and Engineering, University of California, San Diego, La Jolla, USA（计算机科学与工程系，加州大学圣地亚哥分校，拉乔拉，美国）；Halıcıoglu Data Science Institute, University of California, San Diego, La Jolla, USA（Halıcıoglu数据科学研究所，加州大学圣地亚哥分校，拉乔拉，美国）；Scripps Institution of Oceanography, University of California, San Diego, La Jolla, USA（斯克里普斯海洋研究所，加州大学圣地亚哥分校，拉乔拉，美国）
School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院）；University of Chicago（芝加哥大学）
Canonical Correlation Analysis (CCA) is a classical tool for finding correlations among the components of two random vectors. In recent years, CCA has been widely applied to the analysis of genomic data, where it is common for researchers to perform multiple assays on a single set of patient samples. Recent work has proposed sparse variants of CCA to address the high dimensionality of such data. However, classical and sparse CCA are based on linear models, and are thus limited in their ability to find general correlations. In this paper, we present two approaches to high-dimensional nonparamet…
University of Minnesota（明尼苏达大学）；University of Cincinnati（辛辛那提大学）
In many domains, there is significant interest in capturing novel relationships between time series that represent activities recorded at different nodes of a highly complex system. In this paper, we introduce multipoles, a novel class of linear relationships between more than two time series. A multipole is a set of time series that have strong linear dependence among themselves, with the requirement that each time series makes a significant contribution to the linear dependence. We demonstrate that most interesting multipoles can be identified as cliques of negative correlations in a correla…
TU Dortmund（多特蒙德工业大学）；University of Bonn（波恩大学）；University Hospital of Bonn（波恩大学医院）
Not all real-world data are labeled, and when labels are not available, it is often costly to obtain them. Moreover, as many algorithms suffer from the curse of dimensionality, reducing the features in the data to a smaller set is often of great utility. Unsupervised feature selection aims to reduce the number of features, often using feature importance scores to quantify the relevancy of single features to the task at hand. These scores can be based only on the distribution of variables and the quantification of their interactions. The previous literature, mainly investigating anomaly detecti…
Discovering and decoding latent mean-field structure with variational autoencoders
Department of Physics and the James Franck Institute, University of Chicago（芝加哥大学物理系及詹姆斯·弗兰克研究所）；CuspAI；AMLab, University of Amsterdam（阿姆斯特丹大学AMLab）；Leinweber Institute for Theoretical Physics（莱因韦伯理论物理研究所）
Indian Institute of Technology Indore（印度理工学院印多尔分校）；Institute for Basic Science (IBS)（基础科学研究所）；ST Biotherapeutics, Inc.（1ST生物治疗公司）；Indian Institute of Information Technology Raichur（印度赖布尔信息技术学院）
Correlation matrices contain a wide variety of spatio-temporal information about a dynamical system. Predicting correlation matrices from partial time series information of a few nodes characterizes the spatio-temporal dynamics of the entire underlying system. This information can help to predict the underlying network structure, e.g., inferring neuronal connections from spiking data, deducing causal dependencies between genes from expression data, and discovering long spatial range influences in climate variations. Traditional methods of predicting correlation matrices utilize time series dat…
Mila-Quebec AI Institute, Canada（魁北克AI研究所）；Samsung - SAIT AI Lab, Montreal, Canada（三星SAIT人工智能实验室）；University of Montreal, Canada（蒙特利尔大学）；Institute of Science and Technology Austria（奥地利科学与技术研究所）；National University of Singapore（新加坡国立大学）
