通过压缩镜头的统计学习
On statistical learning via the lens of compression
摘要:
这项工作继续研究样本压缩方案和统计学习之间的关系,这主要是在二进制分类的框架内进行了研究。这项工作的中心主题是建立可学习性和可压缩性之间的等价关系,并利用这些等价关系进行统计学习理论的研究。我们从设置多类分类(零/一损失)开始。我们证明,在这种情况下,可学习性是等价的对数样本大小的压缩,一致收敛意味着压缩的恒定大小。然后,我们考虑Vapnik的一般学习设置:我们表明,为了扩展这种情况下的可压缩性,可学习性等价,有必要考虑压缩的近似变体。最后,我们提供了一些应用的可压缩性,可学习性等价:(i)不可知的情况下的可学习性和可实现的情况下的可学习性是等价的多类分类问题(在样本复杂性方面)。(ii)这种不可知情况下的可学习性和可实现情况下的可学习性之间的等价性并不适用于一般的学习问题:存在一个学习问题,其损失函数只取三个值,在此情况下,不可知情况下的可学习性和可实现情况下的可学习性是不等价的。(iii)一致收敛意味着在多类分类问题中压缩恒定大小。部分的参数包括分析的一致收敛速度的图形尺寸,在其中我们改进了以前的界限。(iv)多类分类问题中样本压缩的二分法:如果存在非平凡压缩,则存在对数大小的压缩。(v)多类范畴化问题的一个紧性定理。
英文摘要:
This work continues the study of the relationship between sample compression schemes and statistical learning, which has been mostly investigated within the framework of binary classification. The central theme of this work is establishing equivalences between learnability and compressibility, and utilizing these equivalences in the study of statistical learning theory. We begin with the setting of multiclass categorization (zero/one loss). We prove that in this case learnability is equivalent to compression of logarithmic sample size, and that uniform convergence implies compression of constant size. We then consider Vapnik's general learning setting: we show that in order to extend the compressibility-learnability equivalence to this case, it is necessary to consider an approximate variant of compression. Finally, we provide some applications of the compressibility-learnability equivalences: (i) Agnostic-case learnability and realizable-case learnability are equivalent in multiclass categorization problems (in terms of sample complexity). (ii) This equivalence between agnostic-case learnability and realizable-case learnability does not hold for general learning problems: There exists a learning problem whose loss function takes just three values, under which agnostic-case and realizable-case learnability are not equivalent. (iii) Uniform convergence implies compression of constant size in multiclass categorization problems. Part of the argument includes an analysis of the uniform convergence rate in terms of the graph dimension, in which we improve upon previous bounds. (iv) A dichotomy for sample compression in multiclass categorization problems: If a non-trivial compression exists then a compression of logarithmic size exists. (v) A compactness theorem for multiclass categorization problems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。