arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.03356cs.ITmath.IT
收藏

引入逐点共享信息的可微度量

Introducing a differentiable measure of pointwise shared information

Abdullah Makkeh, Aaron J. Gutknecht, Michael Wibral

摘要:

多元互信息的部分信息分解(PID)描述了一组源变量包含关于目标变量的信息的不同方式。Williams和Beer的开创性工作表明,如果不做额外的假设,这种分解就不能从经典的信息论中确定,并且已经提出了几个候选度量,通常借鉴了决策理论等相关领域的原理。这些措施都是不可微的相对于潜在的概率质量函数。在这里,我们提出了一种新的措施,满足这一属性,出现完全从信息理论的原则,并具有本地互信息的形式。我们展示了如何从概率质量排除的角度来理解该措施,这是Fano的互信息原始定义的基础原则。由于我们的措施是很好地定义为个人实现的随机变量,它本身,例如在人工神经网络的本地学习。我们还表明,它有一个有意义的莫比乌斯反演的冗余格,并遵守目标链规则。我们给出了一个操作的解释措施的基础上,代理人应该采取的决定,如果只提供共享信息。

英文摘要:

Partial information decomposition (PID) of the multivariate mutual information describes the distinct ways in which a set of source variables contains information about a target variable. The groundbreaking work of Williams and Beer has shown that this decomposition cannot be determined from classic information theory without making additional assumptions, and several candidate measures have been proposed, often drawing on principles from related fields such as decision theory. None of these measures is differentiable with respect to the underlying probability mass function. We here present a novel measure that satisfies this property, emerges solely from information-theoretic principles, and has the form of a local mutual information. We show how the measure can be understood from the perspective of exclusions of probability mass, a principle that is foundational to the original definition of the mutual information by Fano. Since our measure is well-defined for individual realizations of the random variables it lends itself for example to local learning in artificial neural networks. We also show that it has a meaningful Möbius inversion on a redundancy lattice and obeys a target chain rule. We give an operational interpretation of the measure based on the decisions that an agent should take if given only the shared information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑