高阶公共信息
Higher-order Common Information
AI总结:
提出高阶公共信息(HCI)概念,通过迭代信息瓶颈构造定义,量化多个随机变量间的共享信息,并给出高斯和伯努利源的闭式解,证明其比现有界限更紧。
摘要:
香农互信息量化了两个随机变量之间的冗余。我们引入了一个新概念,称为高阶公共信息(HCI),它捕捉了$n$个任意分布的随机变量之间共享的信息。该量通过迭代信息瓶颈构造定义,可以解释为单个压缩表示同时保留关于所有变量的信息的最大速率。对于联合高斯和伯努利源,我们推导了任意$n$的闭式表达式。此外,我们证明了HCI比现有界限更严格地表征了冗余,并展示了如何对任意分布的源进行HCI的数值近似。
英文摘要:
Shannon's mutual information quantifies dependence between two random variables. We introduce \emph{higher-order common information} (HCI), a measure of statistical relevance across $n$ random variables. HCI is defined through a sequential information-bottleneck construction in which an auxiliary representation is generated locally from a single reference source and successively optimized for relevance to the remaining sources. The terminal representation is evaluated by its minimum mutual information with any individual source. Thus, an HCI value of $r$ guarantees a locally generated representation having at least $r$ bits of statistical information about every source. We derive closed-form expressions for jointly Gaussian sources and for a binary common-source model for arbitrary finite $n$. For finite-alphabet sources, we further show that the HCI is lower-bounded by the Gács--Körner common information, while it is upper-bounded by the smallest pairwise mutual information. Finally, we propose an HCI-inspired numerical approximation and illustrate its application to multivariate EEG data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。