arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.12767cs.LGcs.AImath.STstat.MLstat.TH
收藏

压缩、泛化与学习

Compression, Generalization and Learning

University of Brescia(布雷西亚大学);Politecnico di Milano(米兰理工大学)

Marco C. Campi, Simone Garatti

AI总结:

本文提出了一种新的压缩变化概率控制理论,证明压缩集基数是压缩变化概率的一致估计量,并在不可知设置下获得了紧致的有限样本界限,为学习中的风险控制和超参数调优提供了理论基础。

摘要:

压缩函数是一种将观测集缩减为较小子集,同时保留其信息内容的映射。在多种应用中,一条新观测使压缩集发生变化的条件被解释为该观测带来了额外信息,在学习理论中,这对应于误分类或误预测。在本文中,我们为一项新理论奠定了基础,该理论允许人们对压缩变化的概率(在学习应用中映射为统计“风险”)保持控制。在适当条件下,压缩集的基数被证明是压缩变化概率的一致估计量(对压缩集的大小没有任何上限要求);此外,在普遍适用的偏好条件下,获得了前所未有的紧致有限样本界限来评估压缩变化的概率。所有结果均可在完全不可知的设置中使用,即不需要对观测的概率分布有任何先验知识。这些结果不仅为发展对观测驱动方法的信任提供了有效支持,还在学习技术中作为超参数调优工具发挥着基础性作用。

英文摘要:

A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change is interpreted that this observation brings in extra information and, in learning theory, this corresponds to misclassification, or misprediction. In this paper, we lay the foundations of a new theory that allows one to keep control on the probability of change of compression (which maps into the statistical "risk" in learning applications). Under suitable conditions, the cardinality of the compressed set is shown to be a consistent estimator of the probability of change of compression (without any upper limit on the size of the compressed set); moreover, unprecedentedly tight finite-sample bounds to evaluate the probability of change of compression are obtained under a generally applicable condition of preference. All results are usable in a fully agnostic setup, i.e., without requiring any a priori knowledge on the probability distribution of the observations. Not only these results offer a valid support to develop trust in observation-driven methodologies, they also play a fundamental role in learning techniques as a tool for hyper-parameter tuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑