arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.01092cs.LGmath.STstat.TH
收藏

压缩系综的统计最优性条件

Statistical optimality conditions for compressive ensembles

Henry W. J. Reeve, Ata Kaban

摘要:

我们提出了一个框架,用于对高维数据的独立随机压缩训练的低复杂性经验风险最小化器的集合进行理论分析。首先,我们引入了一个一般的分布依赖的上限的超额风险,在一个自然的概念的压缩性。这个界限与原始数据表示的维度无关,并且解释了压缩方法的内置正则化效果。然后,我们实例化这个一般绑定到分类和回归任务,考虑约翰逊-林登施特劳斯映射作为压缩方案。对于这些任务中的每一个,我们的策略是开发一个紧密的压缩函数的上限,并通过这样做,我们发现的几何性质的分布条件下,压缩算法达到最小最大最优速率最多的多对数因子。在压缩分类的情况下,这是实现了一个温和的几何边缘条件以及灵活的时刻条件,这是显着更一般的假设有界域。在强凸光滑损失函数回归的情况下,我们发现,压缩回归能够利用光谱衰减与近最佳的保证。此外,我们的中心上限的一个关键因素是一个高概率的统一的上限依赖的经验过程,这可能是独立的利益的综合偏差。

英文摘要:

We present a framework for the theoretical analysis of ensembles of low-complexity empirical risk minimisers trained on independent random compressions of high-dimensional data. First we introduce a general distribution-dependent upper-bound on the excess risk, framed in terms of a natural notion of compressibility. This bound is independent of the dimension of the original data representation, and explains the in-built regularisation effect of the compressive approach. We then instantiate this general bound to classification and regression tasks, considering Johnson-Lindenstrauss mappings as the compression scheme. For each of these tasks, our strategy is to develop a tight upper bound on the compressibility function, and by doing so we discover distributional conditions of geometric nature under which the compressive algorithm attains minimax-optimal rates up to at most poly-logarithmic factors. In the case of compressive classification, this is achieved with a mild geometric margin condition along with a flexible moment condition that is significantly more general than the assumption of bounded domain. In the case of regression with strongly convex smooth loss functions we find that compressive regression is capable of exploiting spectral decay with near-optimal guarantees. In addition, a key ingredient for our central upper bound is a high probability uniform upper bound on the integrated deviation of dependent empirical processes, which may be of independent interest.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑