arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.12192cs.ITmath.IT
收藏

随机链与加强的信息论推广界

Stochastic Chaining and Strengthened Information-Theoretic Generalization Bounds

Ruida Zhou, Chao Tian, Tie Liu

摘要:

我们提出了一种新的方法,应用链接技术结合信息理论的措施,以限制机器学习算法的泛化误差。与Asadi等人先前提出的基于度量空间的分层划分的确定性链接方法不同,我们提出了一种随机链接的方法,它用一个从连续细化信源编码中借用的抽象马尔可夫模型来代替分层划分。这种方法相对于确定性链接有三个好处:1)度量空间不一定是有界的,2)便于后续分析以产生更明确的边界,以及3)通过去除分区的几何刚性来优化边界的进一步机会。所提出的方法包括传统的链接作为一个特殊的情况下,因此也可以利用任何确定性的链接结构。我们说明了这些好处使用的问题估计高斯均值和相位恢复。对于前者,我们推导出一个界,提供了一个有序的改进,在以前的结果,对于后者,我们提供了一个随机链,允许优化的链接参数。

英文摘要:

We propose a new approach to apply the chaining technique in conjunction with information-theoretic measures to bound the generalization error of machine learning algorithms. Different from the deterministic chaining approach based on hierarchical partitions of a metric space, previously proposed by Asadi et al., we propose a stochastic chaining approach, which replaces the hierarchical partitions with an abstracted Markovian model borrowed from successive refinement source coding. This approach has three benefits over deterministic chaining: 1) the metric space is not necessarily bounded, 2) facilitation of subsequent analysis to yield more explicit bound, and 3) further opportunity to optimize the bound by removing the geometric rigidity of the partitions. The proposed approach includes the traditional chaining as a special case, and can therefore also utilize any deterministic chaining construction. We illustrate these benefits using the problem of estimating Gaussian mean and that of phase retrieval. For the former, we derive a bound that provides an order-wise improvement over previous results, and for the latter we provide a stochastic chain that allows optimization over the chaining parameter.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑