arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.07409cs.CL
收藏

DeLS-Spec:用于并行推测性草稿的解耦长短上下文

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics(南京航空航天大学人工智能学院);MIIT Key Laboratory of Pattern Analysis and Machine Intelligence(工业和信息化部模式分析与机器智能重点实验室);The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education(教育部脑机智能技术重点实验室)

Hong-Kai Zheng, Piji Li

AI总结:

研究提出DeLS-Spec解耦长短上下文推测性解码方法,将DFlash模型作长上下文专家,引入轻量级本地头作短上下文专家,可独立训练降低成本,推理时结合二者逻辑,实验显示在多基准测试中比DFlash提升加速比和接受长度。

摘要:

推测性解码通过并行起草多个令牌并进行验证来加速大语言模型推理。像DFlash这样的块并行起草器通过一次性预测整个块进一步提高起草效率,但它们的位置预测缺乏明确的块内因果条件。诸如Domino和DSpark等方法试图将这种因果关系引入块并行起草,但需要从头训练草稿模型,限制了灵活性并增加了训练成本。我们提出了DeLS-Spec,一种解耦长短上下文推测性解码方法。它将固定的DFlash模型视为长上下文专家,并引入轻量级本地头作为短上下文专家。本地头可通过标准的下一个令牌预测目标独立训练,无需与目标模型或DFlash主干联合训练,训练成本极低。在推理时,DeLS-Spec结合长上下文和短上下文逻辑,且本地头不绑定到特定的DFlash检查点,使方法更模块化和灵活。在Qwen3模型上的实验表明,DeLS-Spec在数学、代码和对话基准测试中始终比DFlash提高加速比和平均接受长度。

英文摘要:

Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position-wise predictions lack explicit intra-block causal conditioning. Recent methods such as Domino and DSpark attempt to introduce such causality into block-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost. We propose DeLS-Spec, a decoupled long-short context speculative decoding method. DeLS-Spec treats the fixed DFlash model as a long-context expert and introduces a lightweight local head as a short-context expert. The local head can be trained independently with a standard next-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost. At inference time, DeLS-Spec combines long-context and short-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑