A Class of Nonbinary Symmetric Information Bottleneck Problems

Polar Coding for Achieving the Capacity of Marginal Channels in Nonbinary-Input Setting

Achieving information-theoretic security using explicit coding scheme in which unlimited computational power for eavesdropper is assumed, is one of the main topics is security consideration. It is shown that polar codes are capacity achieving codes and have a low complexity in encoding and decoding. It has been proven that polar codes reach to secrecy capacity in the binary-input wiretap channels in symmetric settings for which the wiretapper's channel is degraded with respect to the main channel. The first task of this paper is to propose a coding scheme to achieve secrecy capacity in asymmet…

数据效率、降维与广义对称信息瓶颈

Data efficiency, dimensionality reduction, and the generalized symmetric information bottleneck

本文提出广义对称信息瓶颈（GSIB），扩展对称信息瓶颈以探索同时压缩的成本形式，并证明同时压缩在数据效率上优于独立压缩，所需数据量更少。

The Compound Information Bottleneck Outlook

We formulate and analyze the compound information bottleneck programming. In this problem, a Markov chain $ \mathsf{X} \rightarrow \mathsf{Y} \rightarrow \mathsf{Z} $ is assumed with fixed marginal distributions $\mathsf{P}_{\mathsf{X}}$ and $\mathsf{P}_{\mathsf{Y}}$, and the mutual information between $ \mathsf{X} $ and $ \mathsf{Z} $ is sought to be maximized over the choice of conditional probability of $\mathsf{Z}$ given $\mathsf{Y}$ from a given class, under the \textit{worst choice} of the joint probability of the pair $(\mathsf{X},\mathsf{Y})$ from a different class. We consider several…

Generalizing Bottleneck Problems

Given a pair of random variables $(X,Y)\sim P_{XY}$ and two convex functions $f_1$ and $f_2$, we introduce two bottleneck functionals as the lower and upper boundaries of the two-dimensional convex set that consists of the pairs $\left(I_{f_1}(W; X), I_{f_2}(W; Y)\right)$, where $I_f$ denotes $f$-information and $W$ varies over the set of all discrete random variables satisfying the Markov condition $W \to X \to Y$. Applying Witsenhausen and Wyner's approach, we provide an algorithm for computing boundaries of this set for $f_1$, $f_2$, and discrete $P_{XY}$. In the binary symmetric case, we f…

Collaborative Information Bottleneck

This paper investigates a multi-terminal source coding problem under a logarithmic loss fidelity which does not necessarily lead to an additive distortion measure. The problem is motivated by an extension of the Information Bottleneck method to a multi-source scenario where several encoders have to build cooperatively rate-limited descriptions of their sources in order to maximize information with respect to other unobserved (hidden) sources. More precisely, we study fundamental information-theoretic limits of the so-called: (i) Two-way Collaborative Information Bottleneck (TW-CIB) and (ii) th…

深度变分多元信息瓶颈——一种变分损失框架

Deep Variational Multivariate Information Bottleneck -- A Framework for Variational Losses

本研究提出深度变分多元信息瓶颈统一框架，推广多种变分降维方法，扩展beta-DVCCA并提出DVSIB新方法，实验验证其在多视图表示学习上的性能优势，为定制损失函数奠定基础。