arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.00769cs.ITmath.IT
收藏

基于缺陷性的多元高斯变量部分信息分解

Partial Information Decomposition via Deficiency for Multivariate Gaussians

Praveen Venkatesh, Gabriel Schamberg

AI总结:

针对多元高斯场景下部分信息分解(PID)计算复杂度高、仅适用于低维的问题,本文刻画了可闭式计算的高斯分布集合,提出基于缺陷性的凸优化近似框架与高效算法,支持数十至数百维变量的PID计算,并验证了近似效果。

摘要:

双变量部分信息分解(PID,Partial Information Decomposition)用于刻画“消息”随机变量中的信息如何在两个“组成”随机变量之间分解为唯一信息、冗余信息和协同信息分量。这些分量是三个变量联合分布的函数,通常通过在所有可能的联合分布空间上进行优化来定义,这导致PID的实际计算面临较大挑战,其应用也被限制在低维随机向量场景。\n为缓解这一计算负担,本文研究联合高斯随机向量的情况。Barrett(2015)曾对该场景展开研究,证明某些具备良好操作动机的PID在标量消息下可简化为闭式表达式。本文指出Barrett的结论通常无法推广到向量消息场景,并刻画了可简化为闭式的多元高斯分布集合。\n针对其余所有多元高斯分布,本文基于统计学中的缺陷性(deficiency)概念,提出一种用于近似计算特定PID定义的凸优化框架。利用高斯场景特有的简化假设,本文给出了一种高效算法,可对维度达数十甚至数百的多元高斯变量近似计算双变量PID,并从理论和实验两方面验证了该近似方法的有效性。

英文摘要:

Bivariate partial information decompositions (PIDs) characterize how the information in a "message" random variable is decomposed between two "constituent" random variables in terms of unique, redundant and synergistic information components. These components are a function of the joint distribution of the three variables, and are typically defined using an optimization over the space of all possible joint distributions. This makes it computationally challenging to compute PIDs in practice and restricts their use to low-dimensional random vectors. To ease this burden, we consider the case of jointly Gaussian random vectors in this paper. This case was previously examined by Barrett (2015), who showed that certain operationally well-motivated PIDs reduce to a closed form expression for scalar messages. Here, we show that Barrett's result does not extend to vector messages in general, and characterize the set of multivariate Gaussian distributions that reduce to closed-form. Then, for all other multivariate Gaussian distributions, we propose a convex optimization framework for approximately computing a specific PID definition based on the statistical concept of deficiency. Using simplifying assumptions specific to the Gaussian case, we provide an efficient algorithm to approximately compute the bivariate PID for multivariate Gaussian variables with tens or even hundreds of dimensions. We also theoretically and empirically justify the goodness of this approximation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑