计算泊松与多项系统的独特信息
Computing Unique Information for Poisson and Multinomial Systems
AI总结:
本文针对泊松和多项系统,给出计算双变量部分信息分解的充分条件,证明独特信息项为零,从而利用边际和联合互信息轻松计算PID。
摘要:
双变量部分信息分解(PID)描述了随机变量M与两个随机变量Y和Z之间的互信息如何分解为独特、冗余和协同项。近来,PID作为理解生物系统和机器学习中偏差的新兴工具显示出潜力。然而,计算PID是一个具有挑战性的问题,因为它通常涉及对分布进行优化。在这项工作中,我们研究了在两种系统中计算PID的问题:受“理想泊松信道”启发的泊松系统和受多项稀疏化启发的多项系统,针对标量M。我们为这两种系统提供了充分条件,在这些条件下,许多基于操作动机的PID可以获得闭式表达式,从而使我们能够轻松计算这些系统的PID。我们的证明包括展示其中一个独特信息项为零,这使得剩余的独特、冗余和协同项可以仅使用边际互信息和联合互信息轻松计算。
英文摘要:
Bivariate Partial Information Decomposition (PID) describes how the mutual information between a random variable M and two random variables Y and Z is decomposed into unique, redundant, and synergistic terms. Recently, PID has shown promise as an emerging tool to understand biological systems and biases in machine learning. However, computing PID is a challenging problem as it typically involves optimizing over distributions. In this work, we study the problem of computing PID in two systems: the Poisson system inspired by the 'ideal Poisson channel' and the multinomial system inspired by multinomial thinning, for a scalar M. We provide sufficient conditions for both systems under which closed-form expressions for many operationally-motivated PID can be obtained, thereby allowing us to easily compute PID for these systems. Our proof consists of showing that one of the unique information terms is zero, which allows the remaining unique, redundant, and synergistic terms to be easily computed using only the marginal and the joint mutual information.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。