二元源信息瓶颈中最优表示的基数
On the Cardinality of Optimal Representations in the Binary-Source Information Bottleneck
查看机构详情
- The University of Hong Kong（香港大学）
- McMaster University（麦克马斯特大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文证明在二元源信息瓶颈中，最优表示可取二元符号，将经典基数上界从|X|+1加强为|X|，并利用分离超平面与熵函数二阶导数比值的凹性完成证明。
中文摘要 AI 辅助
信息瓶颈（IB）寻求源$X$的一个表示$U$，在$I(U;X)$的约束下，尽可能多地保留关于目标$Y$的信息。经典论证表明，考虑至多$|\mathcal{X}|+1$个符号的表示就足够了，并且当$|\mathcal{X}| \geq 3$时，该界是紧的。我们证明二元情形表现不同：若$X$是二元的且$Y$有限，则对于$(X,Y)$的每个联合分布和每个速率约束，IB最优解由二元$U$取得。因此，对于二元源，界$|\mathcal{U}| \leq |\mathcal{X}|+1$可加强为$|\mathcal{U}| \leq |\mathcal{X}|$。证明结合了分离超平面论证与如下观察：对于二元源，所涉及的两个熵函数的二阶导数之比是凹的。
英文摘要
The information bottleneck (IB) seeks a representation $U$ of a source $X$ that retains as much information as possible about a target $Y$, subject to a constraint on $I(U;X)$. A classical argument shows that it suffices to consider representations with at most $|\mathcal{X}|+1$ symbols, and this bound is known to be tight whenever $|\mathcal{X}| \geq 3$. We show that the binary case behaves differently: if $X$ is binary and $Y$ is finite, then for every joint distribution of $(X,Y)$ and every rate constraint, the IB optimum is attained by a binary $U$. Hence the bound $|\mathcal{U}| \leq |\mathcal{X}|+1$ sharpens to $|\mathcal{U}| \leq |\mathcal{X}|$ for binary sources. The proof combines a separating hyperplane argument with the observation that, for a binary source, the ratio of the second derivatives of the two entropy functions involved is concave.
