基于潜在高斯分布中归一化流的偏信息分解
Partial Information Decomposition via Normalizing Flows in Latent Gaussian Distributions
AI总结：
本文提出基于归一化流的高斯偏信息分解（GPID）方法，通过信息保持编码器处理非高斯数据，并设计梯度算法提升计算效率，在合成与多模态基准上验证了其准确性与实用性。
摘要：
多模态研究在多个信息源交互分析能够增强预测建模、数据融合和可解释性的领域中引起了广泛关注。偏信息分解（PID）已成为一种有用的信息论框架，用于量化单个模态独立、冗余或协同地传达关于目标变量信息的程度。然而，现有的PID方法依赖于在由估计的成对概率分布约束的联合分布上进行优化，这对于连续和高维模态而言成本高昂且不准确。我们的第一个关键见解是，当成对分布为多元高斯分布时，该问题可以高效求解，我们将此问题称为高斯PID（GPID）。我们提出了一种新的基于梯度的算法，基于底层优化问题的替代公式，大幅提高了GPID的计算效率。为了将适用性推广到非高斯数据，我们学习信息保持编码器，将任意输入分布的随机变量转换为成对高斯随机变量。在此过程中，我们解决了关于GPID联合高斯解最优性的一个开放问题。在多样合成示例中的实证验证表明，我们提出的方法比现有基线提供了更准确和高效的PID估计。我们进一步评估了一系列大规模多模态基准，以展示其在量化多模态数据集中的PID和选择高性能模型的实际应用中的效用。
英文摘要：
The study of multimodality has garnered significant interest in fields where the analysis of interactions among multiple information sources can enhance predictive modeling, data fusion, and interpretability. Partial information decomposition (PID) has emerged as a useful information-theoretic framework to quantify the degree to which individual modalities independently, redundantly, or synergistically convey information about a target variable. However, existing PID methods depend on optimizing over a joint distribution constrained by estimated pairwise probability distributions, which are costly and inaccurate for continuous and high-dimensional modalities. Our first key insight is that the problem can be solved efficiently when the pairwise distributions are multivariate Gaussians, and we refer to this problem as Gaussian PID (GPID). We propose a new gradient-based algorithm that substantially improves the computational efficiency of GPID based on an alternative formulation of the underlying optimization problem. To generalize the applicability to non-Gaussian data, we learn information-preserving encoders to transform random variables of arbitrary input distributions into pairwise Gaussian random variables. Along the way, we resolved an open problem regarding the optimality of joint Gaussian solutions for GPID. Empirical validation in diverse synthetic examples demonstrates that our proposed method provides more accurate and efficient PID estimates than existing baselines. We further evaluate a series of large-scale multimodal benchmarks to show its utility in real-world applications of quantifying PID in multimodal datasets and selecting high-performing models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
