arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.03554cs.ITmath.ITmath.PR
收藏

部分信息分解的显式公式

Explicit Formula for Partial Information Decomposition

Aobo Lyu, Andrew Clark, Netanel Raviv

AI总结:

本文受 Pearl 的 do-演算启发,引入新的 do-运算,首次给出满足 Williams-Beer 公理的部分信息分解显式公式,解决了该领域长期未决的公开问题。

摘要:

两个随机变量之间的互信息是一个已被充分研究的概念,其理解相当完整。然而,一个随机变量与另外两个随机变量之间的互信息则是一个远为复杂的概念。具体而言,香农互信息无法刻画这三个变量之间的细粒度交互,导致对复杂系统的洞察有限。为了刻画这些细粒度交互,Williams 和 Beer 于 2010 年提出将该互信息分解为信息原子,即唯一信息、冗余信息和协同信息,并提出这些原子必须满足的若干操作性公理。尽管已有大量努力,但迄今仍未找到一个满足这些公理的通用公式。受 Judea Pearl 的 do-演算启发,我们通过引入 do-运算来解决这一公开问题。do-运算是在变量系统上的一种操作,它将某个边际分布设定为期望值,这与任何现有方法都不同。利用该运算,我们给出了第一个显式公式来计算信息原子,从而满足 Williams 和 Beer 的公理,以及该领域后续研究中提出的附加性质。

英文摘要:

Mutual information between two random variables is a well-studied notion, whose understanding is fairly complete. Mutual information between one random variable and a pair of other random variables, however, is a far more involved notion. Specifically, Shannon's mutual information does not capture fine-grained interactions between those three variables, resulting in limited insights in complex systems. To capture these fine-grained interactions, in 2010 Williams and Beer proposed to decompose this mutual information to information atoms, called unique, redundant, and synergistic, and proposed several operational axioms that these atoms must satisfy. In spite of numerous efforts, a general formula which satisfies these axioms has yet to be found. Inspired by Judea Pearl's do-calculus, we resolve this open problem by introducing the do-operation, an operation over the variable system which sets a certain marginal to a desired value, which is distinct from any existing approaches. Using this operation, we provide the first explicit formula for calculating the information atoms so that Williams and Beer's axioms are satisfied, as well as additional properties from subsequent studies in the field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑