用大小不等的样本检验封闭性
Testing Closeness With Unequal Sized Samples
摘要:
我们考虑了两个离散分布的紧密性检验问题,在实际相关的设置\emph{unequal}大小的样本从他们每个人。具体而言,给定目标误差参数$\varepsilon > 0$,$m_1$独立地从未知分布$p,$和$m_2$从未知分布$q$绘制,我们描述了用于区分$p=q$与$||p-q||_1 \geq \varepsilon$的情况的测试。如果$p$和$q$在大多数$n$元素上都得到支持,那么我们的测试成功的概率很高,只要$m_1\geq n^{2/3}/\varepsilon^{4/3}$和$m_2 = Ω(\max\{\frac{n}{\sqrt m_1\varepsilon^2}, \frac{\sqrt n}{\varepsilon^2}\});$,我们就表明这种权衡在整个范围内都是最优的,对于常数因子。这些结果扩展了Chan等人最近的工作,他们建立了两个样本大小相等时的样本复杂度,并通过$n$和$\varepsilon$中的多项式因子收紧了Acharya等人的结果。因此,我们获得了一种用于估计$n$状态上马尔可夫链的混合时间直至$\log n$因子的算法,该算法使用$\tilde{O}(n^{3/2} τ_{mix})$查询到“下一个节点”预言,改进了Batu等人的$\tilde{O}(n^{5/3}τ_{mix})$查询算法。最后,我们注意到我们测试算法的核心是一个相对简单的统计数据,在实践中似乎表现良好,在合成数据和自然语言数据上。
英文摘要:
We consider the problem of closeness testing for two discrete distributions in the practically relevant setting of \emph{unequal} sized samples drawn from each of them. Specifically, given a target error parameter $\varepsilon > 0$, $m_1$ independent draws from an unknown distribution $p,$ and $m_2$ draws from an unknown distribution $q$, we describe a test for distinguishing the case that $p=q$ from the case that $||p-q||_1 \geq \varepsilon$. If $p$ and $q$ are supported on at most $n$ elements, then our test is successful with high probability provided $m_1\geq n^{2/3}/\varepsilon^{4/3}$ and $m_2 = Ω(\max\{\frac{n}{\sqrt m_1\varepsilon^2}, \frac{\sqrt n}{\varepsilon^2}\});$ we show that this tradeoff is optimal throughout this range, to constant factors. These results extend the recent work of Chan et al. who established the sample complexity when the two samples have equal sizes, and tightens the results of Acharya et al. by polynomials factors in both $n$ and $\varepsilon$. As a consequence, we obtain an algorithm for estimating the mixing time of a Markov chain on $n$ states up to a $\log n$ factor that uses $\tilde{O}(n^{3/2} τ_{mix})$ queries to a "next node" oracle, improving upon the $\tilde{O}(n^{5/3}τ_{mix})$ query algorithm of Batu et al. Finally, we note that the core of our testing algorithm is a relatively simple statistic that seems to perform well in practice, both on synthetic data and on natural language data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。