总体恢复的样本复杂性
Sample complexity of population recovery
摘要：
总体恢复问题是指基于不完整或损坏的样本估计分布。考虑一个样本量为$n$的随机调查，调查对象为一群个体，每个被调查者都被要求回答$d$二元问题。我们考虑两个轮询障碍之一：（a）在有损人口恢复中，被轮询者可能以概率$ε$跳过每个问题，（b）在噪声人口恢复中，被轮询者可能以概率$ε$躺在每个问题上。给定$n$有损或噪声样本，目标是以高概率在准确度$δ$内同时估计所有$2^d$二进制向量的概率。本文解决了总体恢复的样本复杂性问题。对于有损模型，最优样本复杂度为$\tildeΘ(δ^{-2\max\{\fracε{1-ε},1\}})$，改进了Moitra和Saks的方法：建立了样本复杂度的下界，改进了样本复杂度的上界，并且最大程度上依赖于维数的对数。令人惊讶的是，当$ε$超过$1/2$时，样本复杂度经历了从参数到非参数速率的相变。对于有噪声的总体恢复，尖锐的样本复杂度变得对维数和尺度更敏感，除了$ε=0,1/2$或$1$的平凡情况之外，作为$\exp(Θ(d^{1/3} \log^{2/3}(1/δ)))$。对于这两种模型，我们的估计量只是计算某个函数的经验均值，这是通过预先求解线性规划（LP）来找到的。奇怪的是，对偶LP可以理解为Le Cam的方法，用于下界极小极大风险，从而建立了所提出的估计量的统计最优性。LP的值由复分析方法确定。
英文摘要：
The problem of population recovery refers to estimating a distribution based on incomplete or corrupted samples. Consider a random poll of sample size $n$ conducted on a population of individuals, where each pollee is asked to answer $d$ binary questions. We consider one of the two polling impediments: (a) in lossy population recovery, a pollee may skip each question with probability $ε$, (b) in noisy population recovery, a pollee may lie on each question with probability $ε$. Given $n$ lossy or noisy samples, the goal is to estimate the probabilities of all $2^d$ binary vectors simultaneously within accuracy $δ$ with high probability. This paper settles the sample complexity of population recovery. For lossy model, the optimal sample complexity is $\tildeΘ(δ^{-2\max\{\fracε{1-ε},1\}})$, improving the state of the art by Moitra and Saks in several ways: a lower bound is established, the upper bound is improved and the result depends at most on the logarithm of the dimension. Surprisingly, the sample complexity undergoes a phase transition from parametric to nonparametric rate when $ε$ exceeds $1/2$. For noisy population recovery, the sharp sample complexity turns out to be more sensitive to dimension and scales as $\exp(Θ(d^{1/3} \log^{2/3}(1/δ)))$ except for the trivial cases of $ε=0,1/2$ or $1$. For both models, our estimators simply compute the empirical mean of a certain function, which is found by pre-solving a linear program (LP). Curiously, the dual LP can be understood as Le Cam's method for lower-bounding the minimax risk, thus establishing the statistical optimality of the proposed estimators. The value of the LP is determined by complex-analytic methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
