Speculative decoding accelerates large language models by verifying drafted tokens in parallel. New draft-training objectives are usually credited with the acceptance they add over cross-entropy on externally written responses, yet many pipelines let the target rewrite them. We ask how much of that gain survives. With a vision-language target and a block drafter, we train three objectives on four versions of one corpus: cross-entropy on corpus tokens, token-level distillation from the target, and distillation gated to positions that would survive verification. Distillation and the gate add large gains on external responses, but after greedy regeneration, distillation adds a smaller gain and the gate little or none. The shrinkage persists across data orders, position weights, verifiers, decoding temperatures and, for distillation, initializations and learning rates. Most of it remains at matched supervised length and at each objective's best measured training length. A second vision corpus shows the shrinkage, as does a text-only replication, for distillation also with an autoregressive drafter. Objective gains should therefore be measured against a target-regenerated baseline and across training lengths. Our code is available at https://github.com/js-lee-AI/TargetRegeneration.
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));South China University of Technology(华南理工大学);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Huawei Technology(华为技术)
AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);Huawei Foundation Model Dept(华为基础模型部)
T2D: Target-to-Draft Model Pruning & Adaptation for Efficient Speculative Decoding
Bellini College of AI, Cybersecurity, and Computing, University of South Florida(南佛罗里达大学贝利尼人工智能、网络安全与计算学院);School of Computing and Augmented Intelligence, Arizona State University(亚利桑那州立大学计算与增强智能学院);University of California, Davis(加州大学戴维斯分校)
Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Key Laboratory of Al Safety, Chinese Academy of Sciences(中国科学院人工智能安全重点实验室)
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
Institute for Artificial Intelligence(人工智能研究院);Department of Computer Science and Technology(计算机科学与技术系);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);ModelBest Inc.(模佰科技);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Tsinghua University(清华大学)
Draft-OPD: On-Policy Distillation for Speculative Draft Models
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Tsinghua University(清华大学);The Chinese University of Hong Kong(香港中文大学);Peking University(北京大学);Zhejiang University(浙江大学)
FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration
EPIC Lab, SJTU(上海交通大学EPIC实验室);UESTC;School of Software Engineering, HUST(华中科技大学软件工程学院);Tsinghua University(清华大学);HKUST(GZ)(香港科技大学(广州));Shanghai AI Laboratory(上海人工智能实验室)
DREAM: Drafting with Refined Target Features and Entropy-Adaptive Cross-Attention Fusion for Multimodal Speculative Decoding
Courant Institute of Mathematical Sciences, New York University(纽约大学数学科学学院);Tandon School of Engineering, New York University(纽约大学工程学院);Cerebras Systems Inc.(Cerebras Systems公司);University of Pennsylvania(宾夕法尼亚大学)
DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics(南京航空航天大学人工智能学院);MIIT Key Laboratory of Pattern Analysis and Machine Intelligence(工业和信息化部模式分析与机器智能重点实验室);The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education(教育部脑机智能技术重点实验室)