Rényi的$α$ -交叉熵的矩阵估计
Estimating Rényi's $α$-Cross-Entropies in a Matrix-Based Way
摘要:
传统的信息论量假设可以访问概率分布。估计这样的分布并不简单。在这里,我们考虑基于函数的交叉熵公式,回避了这种先验估计的要求。在再生核Hilbert空间中,我们给出了Rényi的$α$ -交叉熵的三种度量.每项措施都有其吸引力。我们证明,我们可以估计这些措施在一个无偏的,非参数的,最小最大最优的方式。我们通过样本构造的Gram矩阵来做到这一点。这产生了Rényi的$α$ -交叉熵的基于矩阵的估计。这些估计满足雷尼为发散性建立的所有公理。因此,我们的交叉熵可用于评估分配差异。它们也适用于处理高维分布,因为我们的估计量的收敛速度与样本维数无关。可以在https://github.com/isledge/MBRCE上找到实现这些措施的Python代码
英文摘要:
Conventional information-theoretic quantities assume access to probability distributions. Estimating such distributions is not trivial. Here, we consider function-based formulations of cross entropy that sidesteps this a priori estimation requirement. We propose three measures of Rényi's $α$-cross-entropies in the setting of reproducing-kernel Hilbert spaces. Each measure has its appeals. We prove that we can estimate these measures in an unbiased, non-parametric, and minimax-optimal way. We do this via sample-constructed Gram matrices. This yields matrix-based estimators of Rényi's $α$-cross-entropies. These estimators satisfy all of the axioms that Rényi established for divergences. Our cross-entropies can thus be used for assessing distributional differences. They are also appropriate for handling high-dimensional distributions, since the convergence rate of our estimator is independent of the sample dimensionality. Python code for implementing these measures can be found at https://github.com/isledge/MBRCE
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。