AI 中文总结
针对现有归一化互信息估计器对维度敏感、数值稳定性差的问题，本文提出NMINE神经估计器，结合MINE与MI-NEE方法，在1-8维高斯数据上精度优于KSG基线，为连续多维归一化依赖度量提供新方向。
AI 中文摘要
互信息是一种通用的统计依赖性度量，可捕捉随机变量之间的线性与非线性关系。对于连续多维变量，互信息必须从样本中估计；由于互信息无界，其值无法在不同数据集、维度或应用间直接比较，归一化互信息（Normalized Mutual Information）可将互信息转换为归一化依赖评分，解决该局限。近期研究已证明归一化互信息在分子动力学（arXiv:2405.04980）、可解释机器学习（arXiv:2409.16768）等应用中的实用价值，但现有估计器对维度敏感且数值稳定性差（arXiv:2410.07642）。本文提出一种面向连续变量的全神经归一化互信息估计器，该方法结合基于MINE的神经互信息估计器（arXiv:1801.04062）与受MI-NEE启发的神经边缘熵估计器（arXiv:1905.12957）；互信息采用Donsker–Varadhan表示估计，边缘熵则通过学习各边缘分布与均匀参考分布的散度来估计，进而恢复熵值，所得估计器是基于k近邻的归一化互信息估计（arXiv:2405.04980）的神经替代方案。对1至8维高斯数据的实验表明，所提估计器相比基于KSG的归一化互信息基线估计器，精度有所提升，结果显示神经估计是连续多维场景下归一化依赖度量的有前景方向。
英文摘要
Mutual information is a general measure of statistical dependence that captures both linear and nonlinear relationships between random variables. For continuous and multidimensional variables For continuous multidimensional variables, mutual information must be estimated from samples. Because mutual information is unbounded, its values are not directly comparable across datasets, dimensions, or applications. Normalized mutual information addresses this limitation by converting mutual information into a normalized dependency score. Recent work has demonstrated the practical value of normalized mutual information in applications such as molecular dynamics {arXiv:2405.04980} and interpretable machine learning {arXiv:2409.16768}, but existing estimators remain sensitive to dimensionality and numerical stability {arXiv:2410.07642}. In this paper, we propose a fully neural normalized mutual information estimator for continuous variables. The proposed approach combines a MINE-based neural mutual information estimator {arXiv:1801.04062} with MI-NEE-inspired neural marginal entropy estimators {arXiv:1905.12957}. Mutual information is estimated using the Donsker--Varadhan representation, while marginal entropies are estimated by learning the divergence between each marginal distribution and a uniform reference distribution, from which entropy is recovered. The resulting estimator provides a neural alternative to k-nearest-neighbor-based normalized mutual information estimation {arXiv:2405.04980}. Experiments on Gaussian data from one to eight dimensions show that the proposed estimator improves accuracy over a KSG-based normalized mutual information baseline. These results indicate that neural estimation is a promising direction for normalized dependency measurement in continuous multidimensional settings.
