arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.08003cs.ITmath.IT
收藏

多元信息的广义分解

Generalized Decomposition of Multivariate Information

Thomas F. Varley

AI总结:

本文提出基于 Kullback-Leibler 散度的广义信息分解,消除 PID 的源/目标限制,并揭示协同信息与 TSE 复杂度之间的关系。

摘要:

自提出以来,偏信息分解(PID)已成为一种强大的信息论技术,可用于研究复杂系统中(可能是高阶的)交互结构。尽管其实用性很强,但 PID 的适用性受到两方面限制:需要将元素指定为输入或目标,以及互信息本身的特定结构。本文提出一种广义信息分解,它放宽了源/目标之间的区分,同时仍满足关于信息的基本直觉。该方法基于 Kullback-Leibler 散度的分解,因此能够分析从任意先验更新到任意后验时获得的任何信息。由此,任何可以写成 Kullback-Leibler 散度的信息论度量都可获得 Williams 和 Beer 风格的分解,其中总相关、负熵和互信息均作为特例。本文探讨了这种广义信息分解如何揭示对现有度量以及高阶协同性质的新见解。我们表明,协同信息与著名的 Tononi-Sporns-Edelman(TSE)复杂度密切相关,并且协同信息需要与高 TSE 复杂度相似的整合/分离平衡。最后,本文讨论了该方法如何融入其他推广 PID 的尝试,以及实证应用的可能性。

英文摘要:

Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its utility, the applicability of the PID is restricted by the need to assign elements as either inputs or targets, as well as the specific structure of the mutual information itself. Here, we introduce a generalized information decomposition that relaxes the source/target distinction while still satisfying the basic intuitions about information. This approach is based on the decomposition of the Kullback-Leibler divergence, and consequently allows for the analysis of any information gained when updating from an arbitrary prior to an arbitrary posterior. Consequently, any information-theoretic measure that can be written in as a Kullback-Leibler divergence admits a decomposition in the style of Williams and Beer, including the total correlation, the negentropy, and the mutual information as special cases. In this paper, we explore how the generalized information decomposition can reveal novel insights into existing measures, as well as the nature of higher-order synergies. We show that synergistic information is intimately related to the well-known Tononi-Sporns-Edelman (TSE) complexity, and that synergistic information requires a similar integration/segregation balance as a high TSE complexity. Finally, we end with a discussion of how this approach fits into other attempts to generalize the PID and the possibilities for empirical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑