发表机构
Boson AI; University of Washington; McGill University（Boson AI; 华盛顿大学; 麦吉尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
TreeSpark利用父节点条件分布校准边接受度，实现负载自适应的半自回归投机解码草稿树，在保持无损解码的同时提升接受率和解码速度。
AI 中文摘要
投机解码通过让廉价的草稿模型提出标记，并由目标模型并行验证，从而加速语言模型推理。最近的块状草稿模型使草稿生成几乎免费：单次主干网络前向传播即可生成整个草稿标记块。草稿树有望带来进一步的提升——在一次目标前向传播中验证多个备选续写——但现有的构造方法按逐位置边际概率对候选进行排序，忽略了候选扩展自哪个父节点，因此在半自回归草稿模型上，更宽的树主要增加了排序错误的节点；并且固定大小的树忽略了每个解码轮次和每个服务负载能够支持的投机程度。我们引入了TreeSpark，它从草稿模型现有的马尔可夫头中读取父节点条件分布，成本可忽略不计，将其校准为边接受度估计，并让路径存活率支配其他一切：最佳优先扩展、每轮停止以及负载自适应服务策略。在递归拒绝中，使用无放回采样兄弟节点并匹配残差，确保在任何温度下解码均无损。自适应树在每种温度下都优于匹配的固定预算；与同一草稿模型上调优的链相比，TreeSpark每轮多接受15-25%的草稿标记，单请求墙钟时间解码快8-14%，并且在负载增加时优雅地将树收缩回链。代码和工件：此HTTPS URL
英文摘要
Speculative decoding accelerates language-model inference by letting a cheap drafter propose tokens that the target model verifies in parallel. Recent block drafters make drafting nearly free: a single backbone pass emits an entire block of draft tokens. Draft trees promise a further gain -- several alternative continuations verified in one target forward -- but existing constructions rank candidates by per-position marginals that ignore which parent a candidate extends, so on semi-autoregressive drafters wider trees mostly add mis-ranked nodes; and a tree of fixed size ignores how much speculation each decoding round, and each serving load, can support. We introduce TreeSpark, which reads a parent-conditioned distribution from the drafter's existing Markov head at negligible cost, calibrates it into an edge-acceptance estimate, and lets path survival govern everything else: best-first expansion, per-round stopping, and a load-adaptive serving policy. Sampling siblings without replacement, with matching residuals in recursive rejection, keeps decoding lossless at any temperature. Adaptive trees improve on matched fixed budgets at every temperature; against a tuned chain on the same drafter, TreeSpark accepts 15-25% more draft tokens per round and decodes 8-14% faster in single-request wall-clock, and under rising load it gracefully shrinks the tree back to the chain. Code and artifacts: https://github.com/PopSoda2002/TreeSpark
