arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1507.06296cs.ITmath.IT
收藏

基于相邻事件的互信息界

Mutual Information Bounds via Adjacency Events

Yanjun Han, Or Ordentlich, Ofer Shayevitz

摘要:

两个联合分布随机变量$X$和$Y$之间的互信息是联合分布$P_{XY},$的函数,有时难以处理或估计。$(X,Y)$的统计行为的一个粗略描述是由边缘分布$P_X, P_Y$和由联合分布引起的邻接关系给出的,其中如果$P(x,y)>0$,则$x$和$y$相邻。我们推导出这些实体的互信息的下界。通过将从$X$到$Y$的通道视为一组可能动作的概率分布来获得界限,其中动作确定任何可能输入的输出,并且是独立绘制的。我们还提供了另一种证明基于凸优化,产生一个一般更严格的约束。最后,我们推导出互信息的上界在动作和对$(X,Y)$之间的相邻事件,在这种情况下,一个动作$a$和一对$(x,y)$是相邻的,如果$y=a(x)$。作为一个例子,我们将我们的边界应用到二进制删除通道,并表明,对于特殊情况下的i.i.d.输入分布和删除概率的范围,我们的下限和上限都优于最好的已知的互信息的界限。

英文摘要:

The mutual information between two jointly distributed random variables $X$ and $Y$ is a functional of the joint distribution $P_{XY},$ which is sometimes difficult to handle or estimate. A coarser description of the statistical behavior of $(X,Y)$ is given by the marginal distributions $P_X, P_Y$ and the adjacency relation induced by the joint distribution, where $x$ and $y$ are adjacent if $P(x,y)>0$. We derive a lower bound on the mutual information in terms of these entities. The bound is obtained by viewing the channel from $X$ to $Y$ as a probability distribution on a set of possible actions, where an action determines the output for any possible input, and is independently drawn. We also provide an alternative proof based on convex optimization, that yields a generally tighter bound. Finally, we derive an upper bound on the mutual information in terms of adjacency events between the action and the pair $(X,Y)$, where in this case an action $a$ and a pair $(x,y)$ are adjacent if $y=a(x)$. As an example, we apply our bounds to the binary deletion channel and show that for the special case of an i.i.d. input distribution and a range of deletion probabilities, our lower and upper bounds both outperform the best known bounds for the mutual information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑