使用中间随机变量量化协同信息
Quantifying synergistic information using intermediate stochastic variables
摘要：
随机变量间协同效应的量化是信息论中一个重要的开放性问题。当多个来源一起预测结果变量比单一来源预测的总和更好时，信息协同作用就会发生。它是生物学中的一个基本现象，例如在神经元网络和细胞调节过程中，不同的信息流整合以产生单一响应，但也在社会合作过程以及机器学习的统计推断任务中。在这里，我们提出了一个度量的协同熵和协同信息的第一原则。所提出的措施依赖于所谓的协同随机变量（SRV），其被构造为具有零互信息的个别源变量，但非零互信息的完整的源变量集。我们证明了我们的措施，包括边界和可加性性质的几个基本和期望的属性。此外，我们证明了几个重要的后果，我们的措施，包括不同类型的协同信息可能共存于同一套变量之间的事实。一个数值实现，我们用它来证明，协同作用与弹性噪声。我们的措施可能是一个显着的一步，在多元信息论的研究及其众多的应用。
英文摘要：
Quantifying synergy among stochastic variables is an important open problem in information theory. Information synergy occurs when multiple sources together predict an outcome variable better than the sum of single-source predictions. It is an essential phenomenon in biology such as in neuronal networks and cellular regulatory processes, where different information flows integrate to produce a single response, but also in social cooperation processes as well as in statistical inference tasks in machine learning. Here we propose a metric of synergistic entropy and synergistic information from first principles. The proposed measure relies on so-called synergistic random variables (SRVs) which are constructed to have zero mutual information about individual source variables but non-zero mutual information about the complete set of source variables. We prove several basic and desired properties of our measure, including bounds and additivity properties. In addition, we prove several important consequences of our measure, including the fact that different types of synergistic information may co-exist between the same sets of variables. A numerical implementation is provided, which we use to demonstrate that synergy is associated with resilience to noise. Our measure may be a marked step forward in the study of multivariate information theory and its numerous applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
