发表机构
The University of Tokyo; RIKEN AIP（东京大学; 理化学研究所革新智能统合研究中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对掩码生成模型并行采样中未知依赖问题，提出基于隐藏森林结构的反事实探测采样器，实现亚线性总评估次数与顺序深度，并证明最优性下界。
AI 中文摘要
掩码生成模型提供并行令牌预测，但准确的并行采样必须考虑令牌之间的依赖关系。当依赖关系未知时，寻找安全批次也会耗费模型评估。我们研究包括发现阶段在内的总评估次数能否在序列长度$N$上实现亚线性；亚线性顺序深度随后即可实现。我们考虑具有隐藏森林结构的离散分布，通过固定的近似条件预言机访问。在明确的规律性条件和均匀Hellinger误差界下，对于任意固定目标精度$\varepsilon\in (0,1/8]$和足够大的$N$，我们的采样器实现种子平均总变差误差至多$\varepsilon$，总掩码状态提交次数和顺序深度均以$O(N^C \varepsilon^a)$为界，其中常数$0 <C <1$且$a > 0$。这些保证使用多项式词汇量，并设置由$N$和$\varepsilon$决定的边响应下界。采样器在依赖测试间共享假设揭示的评估，以识别安全并行批次，而无需完全恢复隐藏森林。一个可调参数在探测成本与不可逆提交轮次之间进行权衡。在同一类别中，任何达到相同种子平均精度的可接受不可逆乘积提交采样器，在最坏情况下需要$\Omega(N^c \varepsilon^b)$次反事实提交或提交轮次，其中常数$c,b>0$。
英文摘要
Masked generative models offer parallel token prediction, but accurate parallel sampling must account for dependencies among tokens. When dependencies are unknown, finding safe batches also costs model evaluations. We study whether total evaluations, including discovery, can be sublinear in sequence length $N$; sublinear sequential depth then follows. We consider discrete distributions with hidden forest structure, accessed through a fixed approximate conditional oracle. Under explicit regularity conditions and uniform Hellinger error bounds, for any fixed target accuracy $\varepsilon\in(0,1/8]$ and sufficiently large $N$, our sampler achieves seed-averaged total-variation error at most $\varepsilon$, with total masked-state submissions and sequential depth both bounded by $\widetilde{O}(N^C \varepsilon^{-a})$ for constants $0<C<1$ and $a>0$. These guarantees use polynomial vocabulary size and an edge-response lower bound set by $N$ and $\varepsilon$. The sampler shares evaluations of hypothetical reveals across dependence tests to identify safe parallel batches without requiring full recovery of the hidden forest. A tunable parameter trades probing cost against irreversible commit rounds. In the same class, any admissible irreversible product-commit sampler attaining the same seed-averaged accuracy requires $Ω(N^c \varepsilon^b)$ counterfactual submissions or commit rounds in the worst case, for constants $c,b>0$.
