arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1611.04035cs.AIcs.ITmath.ITstat.ML
收藏

熵因果推理

Entropic Causal Inference

The University of Texas at Austin(德克萨斯大学奥斯汀分校);California Institute of Technology(加州理工学院)

Murat Kocaoglu, Alexandros G. Dimakis, Sriram Vishwanath, Babak Hassibi

摘要:

本文研究了利用观测数据确定两个离散随机变量之间因果关系方向的问题。与以前的工作不同,我们保留了最一般的函数模型,但对未观察到的外生变量做了一个假设:受奥卡姆剃刀的启发,我们假设外生变量在真正的因果方向上是简单的。我们用雷尼熵来量化简单性。我们的主要结果是,在自然假设下,如果外生变量在真实方向上具有低$H_0$熵(基数),则它在错误方向上必须具有高$H_0$熵。建立了估计最小熵外生变量的几个算法困难性结果。我们证明了寻找具有最小熵的外生变量的问题等价于寻找给定$n$边缘分布的最小联合熵的问题,也称为最小熵耦合问题。我们提出了一个有效的贪心算法的最小熵耦合问题,即$n=2$可证明找到一个局部最优。这给出了一个贪心算法来寻找具有最小$H_1$(香农熵)的外生变量。我们的贪婪熵为基础的因果推理算法具有类似的性能,最先进的加性噪声模型在真实的数据集。我们的方法的一个优点是,我们不使用随机变量的值,但只有他们的分布。因此,我们的方法可以用于因果推理的顺序和分类数据,不像加性噪声模型。

英文摘要:

We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the unobserved exogenous variable: Inspired by Occam's razor, we assume that the exogenous variable is simple in the true causal direction. We quantify simplicity using Rényi entropy. Our main result is that, under natural assumptions, if the exogenous variable has low $H_0$ entropy (cardinality) in the true direction, it must have high $H_0$ entropy in the wrong direction. We establish several algorithmic hardness results about estimating the minimum entropy exogenous variable. We show that the problem of finding the exogenous variable with minimum entropy is equivalent to the problem of finding minimum joint entropy given $n$ marginal distributions, also known as minimum entropy coupling problem. We propose an efficient greedy algorithm for the minimum entropy coupling problem, that for $n=2$ provably finds a local optimum. This gives a greedy algorithm for finding the exogenous variable with minimum $H_1$ (Shannon Entropy). Our greedy entropy-based causal inference algorithm has similar performance to the state of the art additive noise models in real datasets. One advantage of our approach is that we make no use of the values of random variables but only their distributions. Our method can therefore be used for causal inference for both ordinal and also categorical data, unlike additive noise models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑