arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.09442cs.DScs.AIcs.LGmath.PR
收藏

基于计数的并行采样

Parallel Sampling via Counting

Stanford University(斯坦福大学)

Nima Anari, Ruiquan Gao, Aviad Rubinstein

AI总结:

本研究提出一种基于计数查询预言机的并行采样算法,实现任意分布下关于$n$次线性的并行运行时间,为自回归模型采样带来约$n^{1/3}$倍加速可能,并证明了对应算法的运行时间下界。

摘要:

我们展示了如何利用并行化加速从乘积空间$[q]^n$上的任意分布$μ$中采样,前提是可通过预言机访问计数查询:对任意$S\subseteq [n]$和$σ_S \in [q]^S$,查询$\mathbb{P}_{X\sim μ}[X_S=σ_S]$。我们的算法并行时间复杂度为$O({n^{2/3}\cdot \operatorname{polylog}(n,q)})$,据我们所知,这是首个针对任意分布、运行时间关于$n$次线性的算法。我们的结果对自回归模型中的采样具有启示意义。我们的算法可直接适配回答条件边缘查询$\mathbb{P}_{X\sim μ}[X_i=σ_i\\;\vert\\; X_S=σ_S]$的等价预言机,在自回归模型中该预言机由训练好的神经网络充当。这表明任意阶自回归模型的采样有望实现约$n^{1/3}$倍的加速。我们还补充了正面结果,证明了任何对计数预言机查询次数不超过$\operatorname{poly}(n)$的并行采样算法,即使在$q=2$的情况下,运行时间下界也为$\widetildeΩ(n^{1/3})$。

英文摘要:

We show how to use parallelization to speed up sampling from an arbitrary distribution $μ$ on a product space $[q]^n$, given oracle access to counting queries: $\mathbb{P}_{X\sim μ}[X_S=σ_S]$ for any $S\subseteq [n]$ and $σ_S \in [q]^S$. Our algorithm takes $O({n^{2/3}\cdot \operatorname{polylog}(n,q)})$ parallel time, to the best of our knowledge, the first sublinear in $n$ runtime for arbitrary distributions. Our results have implications for sampling in autoregressive models. Our algorithm directly works with an equivalent oracle that answers conditional marginal queries $\mathbb{P}_{X\sim μ}[X_i=σ_i\;\vert\; X_S=σ_S]$, whose role is played by a trained neural network in autoregressive models. This suggests a roughly $n^{1/3}$-factor speedup is possible for sampling in any-order autoregressive models. We complement our positive result by showing a lower bound of $\widetildeΩ(n^{1/3})$ for the runtime of any parallel sampling algorithm making at most $\operatorname{poly}(n)$ queries to the counting oracle, even for $q=2$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑