信息瓶颈法
The information bottleneck method
摘要：
我们将信号$x\in X$中的相关信息定义为该信号提供的关于另一信号$y\in \Y$的信息。例子包括面部图像提供的关于所描绘的人的名字的信息，或者语音提供的关于所说的话的信息。理解信号$x$不仅需要预测$y$，还需要指定$\X$的哪些特征在预测中起作用。我们将这个问题形式化为找到一个保留有关$\Y$的最大信息的$\X$的短代码。也就是说，我们通过由有限的码字集$\tX$形成的“瓶颈”挤压$\X$提供的关于$\Y$的信息。该约束优化问题可以被看作是率失真理论的推广，其中失真度量$d(x,\x)$从$\X$和$\Y$的联合统计中出现。这种方法产生一组精确的自洽方程的编码规则$X \to \tX$和$\tX \to \Y$。这些方程的解可以通过将Blahut-Arimoto算法推广的收敛重新估计方法来找到。我们的变分原理为讨论信号处理和学习中的各种问题提供了令人惊讶的丰富框架，正如将在其他地方详细描述的那样。
英文摘要：
We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide about the names of the people portrayed, or the information that speech sounds provide about the words spoken. Understanding the signal $x$ requires more than just predicting $y$, it also requires specifying which features of $\X$ play a role in the prediction. We formalize this problem as that of finding a short code for $\X$ that preserves the maximum information about $\Y$. That is, we squeeze the information that $\X$ provides about $\Y$ through a `bottleneck' formed by a limited set of codewords $\tX$. This constrained optimization problem can be seen as a generalization of rate distortion theory in which the distortion measure $d(x,\x)$ emerges from the joint statistics of $\X$ and $\Y$. This approach yields an exact set of self consistent equations for the coding rules $X \to \tX$ and $\tX \to \Y$. Solutions to these equations can be found by a convergent re-estimation method that generalizes the Blahut-Arimoto algorithm. Our variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
