arXiv 1901.075302019-01-19cs.ITcs.DSmath.IT

最小熵耦合及其应用

Minimum--Entropy Couplings and their Applications

Ferdinando Cicalese, Luisa Gargano, Ugo Vaccaro

备注：: This paper has been accepted for publication in IEEE Transactions on Information Theory. arXiv admin note: text overlap with arXiv:1701.05243

摘要：

给定两个离散随机变量$X$和$Y,$，分别具有概率分布${\bf p}=(p_1, \ldots , p_n)$和${\bf q}=(q_1, \ldots , q_m)$，用${\cal C}({\bf p}, {\bf q})$表示${\bf p}$和${\bf q}$的所有耦合的集合，即，具有${\bf p}$和${\bf q}$作为边缘的所有二元概率分布的集合。在本文中，我们研究的问题，找到一个联合概率分布在${\cal C}({\bf p}, {\bf q})$的\emph{minimum entropy}（相当于，耦合，\emph{maximizes}之间的交互信息$X$和$Y$），我们讨论了几种情况下，需要这种优化自然出现。由于优化问题是NP难的，我们给出了一个有效的算法来找到一个联合概率分布在${\cal C}({\bf p}, {\bf q})$的熵超过最小可能的最多{1位}，从而提供了一个近似算法的添加剂的差距最多为1位。利用这个算法，我们将我们的结果推广到寻找任意$k\geq 2$离散随机变量$X_1, \ldots , X_k$的最小熵联合分布的问题，与已知的单个随机变量$X_1, \ldots , X_k$的$k$边缘分布一致。在这种情况下，我们的算法与最优值有{最多$\log k$的加性差距}。我们还讨论了几个相关的应用，我们的研究结果和{扩展我们的结果熵不同的香农熵。

英文摘要：

Given two discrete random variables $X$ and $Y,$ with probability distributions ${\bf p}=(p_1, \ldots , p_n)$ and ${\bf q}=(q_1, \ldots , q_m)$, respectively, denote by ${\cal C}({\bf p}, {\bf q})$ the set of all couplings of ${\bf p}$ and ${\bf q}$, that is, the set of all bivariate probability distributions that have ${\bf p}$ and ${\bf q}$ as marginals. In this paper, we study the problem of finding a joint probability distribution in ${\cal C}({\bf p}, {\bf q})$ of \emph{minimum entropy} (equivalently, a coupling that \emph{maximizes} the mutual information between $X$ and $Y$), and we discuss several situations where the need for this kind of optimization naturally arises. Since the optimization problem is known to be NP-hard, we give an efficient algorithm to find a joint probability distribution in ${\cal C}({\bf p}, {\bf q})$ with entropy exceeding the minimum possible at most by {1 bit}, thus providing an approximation algorithm with an additive gap of at most 1 bit. Leveraging on this algorithm, we extend our result to the problem of finding a minimum--entropy joint distribution of arbitrary $k\geq 2$ discrete random variables $X_1, \ldots , X_k$, consistent with the known $k$ marginal distributions of the individual random variables $X_1, \ldots , X_k$. In this case, our algorithm has an { additive gap of at most $\log k$ from optimum.} We also discuss several related applications of our findings and {extensions of our results to entropies different from the Shannon entropy.}

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文