通过平滑分析实现更快的PAC学习和更小的核心集
Faster PAC Learning and Smaller Coresets via Smoothed Analysis
摘要：
PAC学习通常旨在从$n$项中计算一个小子集（$\varepsilon$ -样本/网络），该子集可证明地近似来自给定查询集合的每个查询（模型，分类器，假设）的给定损失函数，直到加性误差$\varepsilon\in(0,1)$。核心集将此思想推广到支持乘法错误$1\pm\varepsilon$。受平滑分析的启发，我们提出了一个自然的概括：近似查询的\emph{average}（而不是最坏情况）错误，希望得到更小的子集。不同查询的错误之间的依赖性意味着我们可以不再对固定的查询应用Boff-Hoeffding不等式，然后使用VC维或联合界。本文提供了确定性和随机算法计算这样的coresets和$\varepsilon$ -样本的大小独立于$n$，任何有限的查询集和损失函数。示例应用包括新的和改进的核心集构造，例如流向量摘要[ICML'17]和$k$ -PCA [NIPS'16]。实验结果与开放源代码提供。
英文摘要：
PAC-learning usually aims to compute a small subset ($\varepsilon$-sample/net) from $n$ items, that provably approximates a given loss function for every query (model, classifier, hypothesis) from a given set of queries, up to an additive error $\varepsilon\in(0,1)$. Coresets generalize this idea to support multiplicative error $1\pm\varepsilon$. Inspired by smoothed analysis, we suggest a natural generalization: approximate the \emph{average} (instead of the worst-case) error over the queries, in the hope of getting smaller subsets. The dependency between errors of different queries implies that we may no longer apply the Chernoff-Hoeffding inequality for a fixed query, and then use the VC-dimension or union bound. This paper provides deterministic and randomized algorithms for computing such coresets and $\varepsilon$-samples of size independent of $n$, for any finite set of queries and loss function. Example applications include new and improved coreset constructions for e.g. streaming vector summarization [ICML'17] and $k$-PCA [NIPS'16]. Experimental results with open source code are provided.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
