使用学习表示近似高维变量的互信息
Approximating mutual information of high-dimensional variables using learned representations
AI总结:
提出 latent MI (LMI) 方法,利用低维表示近似高维变量互信息,在依赖结构具低内在维度时可处理 $> 10^3$ 维变量,并成功应用于蛋白质相互作用和细胞命运信息量化。
摘要:
互信息(MI)是统计依赖性的通用度量,在科学领域有广泛应用。然而,估计多维变量间的 MI 具有挑战性,因为收敛到准确估计所需的样本数量随维度不利地增长。在实践中,现有技术能可靠估计数十维的 MI,但在更高维度下会失败,因为足够的样本量是不可行的。在此,我们探索利用高维数据中潜在的低维结构,在现实样本量下忠实地近似高维环境中的 MI 的想法。我们开发了一种称为 latent MI (LMI) 近似的方法,该方法将非参数 MI 估计器应用于由简单且具理论动机的模型架构学习的低维表示上。使用多个基准测试,我们表明与现有技术不同,如果变量的依赖结构具有低内在维度,LMI 能很好地近似 $> 10^3$ 维变量的 MI。最后,我们在两个生物学开放问题上展示了 LMI。首先,我们近似相互作用蛋白质的蛋白质语言模型(pLM)表示间的 MI,发现 pLM 编码了关于蛋白质-蛋白质相互作用的非平凡信息。其次,我们量化了造血干细胞单细胞 RNA-seq (scRNA-seq) 测量中包含的细胞命运信息,发现在中性粒细胞分化过程中存在急剧转变,此时 scRNA-seq 捕获的命运信息急剧增加。
英文摘要:
Mutual information (MI) is a general measure of statistical dependence with widespread application across the sciences. However, estimating MI between multi-dimensional variables is challenging because the number of samples necessary to converge to an accurate estimate scales unfavorably with dimensionality. In practice, existing techniques can reliably estimate MI in up to tens of dimensions, but fail in higher dimensions, where sufficient sample sizes are infeasible. Here, we explore the idea that underlying low-dimensional structure in high-dimensional data can be exploited to faithfully approximate MI in high-dimensional settings with realistic sample sizes. We develop a method that we call latent MI (LMI) approximation, which applies a nonparametric MI estimator to low-dimensional representations learned by a simple, theoretically-motivated model architecture. Using several benchmarks, we show that unlike existing techniques, LMI can approximate MI well for variables with $> 10^3$ dimensions if their dependence structure has low intrinsic dimensionality. Finally, we showcase LMI on two open problems in biology. First, we approximate MI between protein language model (pLM) representations of interacting proteins, and find that pLMs encode non-trivial information about protein-protein interactions. Second, we quantify cell fate information contained in single-cell RNA-seq (scRNA-seq) measurements of hematopoietic stem cells, and find a sharp transition during neutrophil differentiation when fate information captured by scRNA-seq increases dramatically.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。