Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this is true of all such Shannon information measures when used to analyze multivariate dependencies. This has broad implications, particularly when employing information to express the organization and mechanisms embedded in complex systems, including the burgeoning efforts to combine complex network theory with information theory. Here, we do not suggest that any aspect of information theory is wrong. Rather, the vast majority of its informational measures are simply inadequate for determining the meaningful dependency structure within joint probability distributions. Therefore, such information measures are inadequate for discovering intrinsic causal relations. We close by demonstrating that such distributions exist across an arbitrary set of variables.
Shannon invariants: A scalable approach to information decomposition
Georg-August University Göttingen（哥廷根大学）；Institute for Dynamics of Biological Networks（生物网络动力学研究所）；Max Planck Institute for Dynamics and Self-Organization（动态与自组织Max Planck研究所）；University of Sussex（Sussex大学）；Department of Informatics（信息学院）；Imperial College London（伦敦帝国学院）；University College London（伦敦大学学院）；University of Oxford（牛津大学）；Principles of Intelligent Behaviour in Biological and Social Systems (PIBBSS)（生物和社会系统智能行为原理（PIBBSS））
The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal
在多元系统中获得有意义的统计依赖性的定量描述是一个困难的开放性问题。部分信息分解（Partial Information Decomposition，PID）是将目标变量的互信息（Mutual Information，MI）分解为预测变量的不同子集中冗余、唯一和协同的分量。在这里，我们建议将PID的优雅形式主义应用于多变量熵，从而产生部分熵分解（PED）。我们实现了PED的熵冗余度测量的基础上逐点常见的代数，一个自然的定义，这是密切相关的MI的定义。我们展示了这种方法可以揭示二元与三元生成结构的多变量系统是无法区分的经典香农措施。熵的观点还表明，错误信息是协同熵，因此，MI本身包括冗余和协同效应。我们在两个预测PED和MI之间的关系，并推导出两种替代的信息分解，我们说明了几个示例系统。这表明，在熵方面，单变量预测MI是不是一个适当的子集的联合MI，我们建议这个以前未认识到的事实部分解释了为什么获得一致的PID已被证明是困难的。PED还允许单独量化机械冗余（与系统功能相关）与源冗余（由输入之间的依赖关系引起）;这是一个现有方法无法解决的重要区别。PED提供的新视角有助于澄清PID方法遇到的一些困难，由此产生的分解为广泛的应用领域的实际数据分析提供了有用的工具。
Semantic Communications: Principles and Challenges
Queen Mary University of London（伦敦玛丽女王大学）；Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Huawei Technologies（华为技术有限公司）；Imperial College London（帝国理工学院）
