基于划分模型贝叶斯比较的相互独立性模式自动提取

Nonparametric independence testing via mutual information

We propose a test of independence of two multivariate random vectors, given a sample from the underlying population. Our approach, which we call MINT, is based on the estimation of mutual information, whose decomposition into joint and marginal entropies facilitates the use of recently-developed efficient entropy estimators derived from nearest neighbour distances. The proposed critical values, which may be obtained from simulation (in the case where one marginal is known) or resampling, guarantee that the test has nominal size, and we provide local power analyses, uniformly over classes of de…

Distance Correlation Sure Independence Screening for Accelerated Feature Selection in Parkinson's Disease Vocal Data

With the abundance of machine learning methods available and the temptation of using them all in an ensemble method, having a model-agnostic method of feature selection is incredibly alluring. Principal component analysis was developed in 1901 and has been a strong contender in this role since, but in the end is an unsupervised method. It offers no guarantee that the features that are selected have good predictive power because it does not know what is being predicted. To this end, Peng et al. developed the minimum redundancy-maximum relevance (mRMR) method in 2005. It uses the mutual informat…

Fast Fair Regression via Efficient Approximations of Mutual Information

Most work in algorithmic fairness to date has focused on discrete outcomes, such as deciding whether to grant someone a loan or not. In these classification settings, group fairness criteria such as independence, separation and sufficiency can be measured directly by comparing rates of outcomes between subpopulations. Many important problems however require the prediction of a real-valued outcome, such as a risk score or insurance premium. In such regression settings, measuring group fairness criteria is computationally challenging, as it requires estimating information-theoretic divergences b…

学习独立性检验的表示

Learning Representations for Independence Testing

本文研究了通过变分估计互信息学习强大独立性检验的方法，探讨了变分互信息测试与HSIC测试的联系，并提出神经依赖统计量(NDS)以提升检验效能。

Learning non-parametric Markov networks with mutual information

We propose a method for learning Markov network structures for continuous data without invoking any assumptions about the distribution of the variables. The method makes use of previous work on a non-parametric estimator for mutual information which is used to create a non-parametric test for multivariate conditional independence. This independence test is then combined with an efficient constraint-based algorithm for learning the graph structure. The performance of the method is evaluated on several synthetic data sets and it is shown to learn considerably more accurate structures than compet…

Conditional independence testing based on a nearest-neighbor estimator of conditional mutual information

Conditional independence testing is a fundamental problem underlying causal discovery and a particularly challenging task in the presence of nonlinear and high-dimensional dependencies. Here a fully non-parametric test for continuous data based on conditional mutual information combined with a local permutation scheme is presented. Through a nearest neighbor approach, the test efficiently adapts also to non-smooth distributions due to strongly nonlinear dependencies. Numerical experiments demonstrate that the test reliably simulates the null distribution even for small sample sizes and with hi…

通过神经上下文分解发现连续变量上的局部独立性

On Discovery of Local Independence over Continuous Variables via Neural Contextual Decomposition

针对连续变量上难以检验上下文特定独立性的问题，本文提出 CSSI 概念与神经上下文分解 NCD，通过学习联合结果空间划分发现局部因果独立性。

使用更少条件独立性检验的因果发现

Causal Discovery with Fewer Conditional Independence Tests

针对约束式因果发现算法需指数级条件独立性检验的问题，提出用多项式数量检验学习因果一致划分图（CCPG），在可识别情况下可还原真实因果图。

Potential Conditional Mutual Information: Estimators, Properties and Applications

The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model inference, causal strength estimation and time-series problems. In several applications, it is desirable to have a functional purely of the conditional distribution p_{Y|X,Z} rather than of the joint distribution p_{X,Y,Z}. We define the potential conditional mutual information as the conditional mutual information calculated with a modified joint distribut…

Generalizing Distance Covariance to Measure and Test Multivariate Mutual Dependence

We propose three measures of mutual dependence between multiple random vectors. All the measures are zero if and only if the random vectors are mutually independent. The first measure generalizes distance covariance from pairwise dependence to mutual dependence, while the other two measures are sums of squared distance covariance. All the measures share similar properties and asymptotic distributions to distance covariance, and capture non-linear and non-monotone mutual dependence between the random vectors. Inspired by complete and incomplete V-statistics, we define the empirical measures and…