可伸缩向量高斯信息瓶颈
Scalable Vector Gaussian Information Bottleneck
摘要：
在统计学习的背景下，信息瓶颈方法通过在压缩复杂度（由最小描述长度测量）和在对数损失度量下评估的失真之间进行适当的权衡来寻求准确性和泛化能力之间的正确平衡。在本文中，我们研究的问题，称为可扩展的信息瓶颈，其中编码器输出多个描述的观察越来越丰富的功能的变化。该模型，这是连续的细化类型与降级的边信息流在解码器，是由一些应用场景，需要不同的准确性水平取决于允许的（或目标）的复杂性水平的动机。我们建立了一个分析表征的最佳相关性复杂度区域的矢量高斯源。然后，我们推导出一个变分推理型算法的一般来源未知的分布，并显示使用神经网络参数化的手段。最后，我们提供的MNIST数据集上的实验结果表明，该方法更好地推广到看不见的数据在训练阶段。
英文摘要：
In the context of statistical learning, the Information Bottleneck method seeks a right balance between accuracy and generalization capability through a suitable tradeoff between compression complexity, measured by minimum description length, and distortion evaluated under logarithmic loss measure. In this paper, we study a variation of the problem, called scalable information bottleneck, in which the encoder outputs multiple descriptions of the observation with increasingly richer features. The model, which is of successive-refinement type with degraded side information streams at the decoders, is motivated by some application scenarios that require varying levels of accuracy depending on the allowed (or targeted) level of complexity. We establish an analytic characterization of the optimal relevance-complexity region for vector Gaussian sources. Then, we derive a variational inference type algorithm for general sources with unknown distribution; and show means of parametrizing it using neural networks. Finally, we provide experimental results on the MNIST dataset which illustrate that the proposed method generalizes better to unseen data during the training phase.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
