抽样估计任意子集和
Sampling to estimate arbitrary subset sums
摘要：
从一组加权项开始，我们希望创建一个特定大小的通用样本，以便稍后用于估计任意子集的总权重。为此，我们提出了优先级抽样测试互联网上的数据比以前的方法的数量级更好。优先级采样的定义和实现都很简单：我们考虑一系列项目i=0，...，n-1，权重为w_i。对于每个项目i，我们在（0，1）中生成一个随机数r_i，并创建一个优先级q_i=w_i/r_i。样本S由k个最高优先级的项目组成。设t为第（k+1）个最高优先级。S中的每个采样项i得到权重估计W_i=max{w_i，t}，而非采样项得到权重估计W_i=0。神奇的是，结果证明权重估计是无偏的，即E[W_i]=w_i，并且通过期望的线性，我们只需将来自子集的采样权重估计相加就可以得到任何子集和的无偏估计。我们还可以估计的方差，令人惊讶的是，不同的权重估计值W_i和W_j之间没有协方差。我们推测一个非常强的近似最优性，即对于任何权重序列，不存在专门的方案，用无偏估计对k个项目进行抽样，得到比k+1个项目的优先抽样更小的总方差。最近，马里奥·塞格迪解决了这个猜想。
英文摘要：
Starting with a set of weighted items, we want to create a generic sample of a certain size that we can later use to estimate the total weight of arbitrary subsets. For this purpose, we propose priority sampling which tested on Internet data performed better than previous methods by orders of magnitude. Priority sampling is simple to define and implement: we consider a steam of items i=0,...,n-1 with weights w_i. For each item i, we generate a random number r_i in (0,1) and create a priority q_i=w_i/r_i. The sample S consists of the k highest priority items. Let t be the (k+1)th highest priority. Each sampled item i in S gets a weight estimate W_i=max{w_i,t}, while non-sampled items get weight estimate W_i=0. Magically, it turns out that the weight estimates are unbiased, that is, E[W_i]=w_i, and by linearity of expectation, we get unbiased estimators over any subset sum simply by adding the sampled weight estimates from the subset. Also, we can estimate the variance of the estimates, and surpricingly, there is no co-variance between different weight estimates W_i and W_j. We conjecture an extremely strong near-optimality; namely that for any weight sequence, there exists no specialized scheme for sampling k items with unbiased estimators that gets smaller total variance than priority sampling with k+1 items. Very recently Mario Szegedy has settled this conjecture.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
