互信息的分布
Distribution of Mutual Information
摘要:
具有联合概率t_ij的两个随机变量i和j的互信息通常用于学习贝叶斯网以及许多其他领域。机会t_ij通常通过经验采样频率n_ij/n来估计,从而导致互信息的点估计I(n_ij/n)。为了回答像“I(n_ij/n)与零一致吗?或者“真实的互信息比点估计大得多的概率是多少?“我们必须超越点估计。在贝叶斯框架中,可以通过利用包括关于t的先验信息的(二阶)先验分布p(t)来回答这些问题。根据先验p(t),可以计算后验p(t| n),由此得到分布p(I| n)的互信息。我们得到可靠的和快速计算的近似p(I| n)。我们专注于均值,方差,偏度,峰度和非信息先验。对于平均值,我们也给出了精确的表达式。数值问题和有效性的范围进行了讨论。
英文摘要:
The mutual information of two random variables i and j with joint probabilities t_ij is commonly used in learning Bayesian nets as well as in many other fields. The chances t_ij are usually estimated by the empirical sampling frequency n_ij/n leading to a point estimate I(n_ij/n) for the mutual information. To answer questions like "is I(n_ij/n) consistent with zero?" or "what is the probability that the true mutual information is much larger than the point estimate?" one has to go beyond the point estimate. In the Bayesian framework one can answer these questions by utilizing a (second order) prior distribution p(t) comprising prior information about t. From the prior p(t) one can compute the posterior p(t|n), from which the distribution p(I|n) of the mutual information can be calculated. We derive reliable and quickly computable approximations for p(I|n). We concentrate on the mean, variance, skewness, and kurtosis, and non-informative priors. For the mean we also give an exact expression. Numerical issues and the range of validity are discussed.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。