Multiple heads decoding accelerates the inference of Large Language Models (LLMs) by predicting next several tokens simultaneously. It generates and verifies multiple candidate sequences in parallel via tree attention with a fixed structure. In this paper, we replace the fixed tree attention with dynamic tree attention on multiple head decoding, specifically in the context of MEDUSA. We propose a simple and low complexity strategy to generate candidates and construct the dynamic tree structure. Preliminary experiments show that the proposed method improves the decoding efficiency of multiple head decoding for LLMs while maintaining the generation quality. This result demonstrates the potential for improvement of multiple head decoding in candidate generation.
DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations
University of Edinburgh（爱丁堡大学）；Centre for AI, Data Science & Artificial Intelligence, R&D, AstraZeneca（阿斯利康研发中心人工智能、数据科学与人工智能中心）；University College London（伦敦大学学院）
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；Center for Data Science, AAIS, Peking University（北京大学AAIS数据科学中心）；Meituan（美团）；Meta AI；National Key Laboratory of General Artificial Intelligence（通用人工智能全国重点实验室）；BIGAI（北京智源人工智能研究院）
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；South China University of Technology（华南理工大学）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Huawei Technology（华为技术）
Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Key Laboratory of Al Safety, Chinese Academy of Sciences（中国科学院人工智能安全重点实验室）
Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
Advanced Micro Devices, Inc.（超威半导体公司）；Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University（西安交通大学人工智能与机器人研究所）；University of HongKong（香港大学）
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Princeton University（普林斯顿大学）；Together AI；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）；University of Connecticut（康涅狄格大学）
Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding
Advanced Micro Devices, Inc., Beijing, China（Advanced Micro Devices公司，北京）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系）；Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）
Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院）；AI Platform, Xiaohongshu Inc.（小红书人工智能平台）；University of Electronic Science and Technology of China（电子科技大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
SimSD: Simple Speculative Decoding in Diffusion Language Models
University of California San Diego（加州大学圣地亚哥分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Google（谷歌）；University of California Santa Barbara（加州大学圣芭芭拉分校）
PIC 4th Challenge: Semantic-Assisted Multi-Feature Encoding and Multi-Head Decoding for Dense Video Captioning
University of Chinese Academy of Sciences（中国科学院大学）；CASIA（中国科学院自动化研究所）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decoding
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（教育部多媒体可信感知与高效计算重点实验室，厦门大学）；Shandong Inspur Database Technology Co.,Ltd（山东浪潮数据库技术有限公司）
SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image Generation
The University of Hong Kong（香港大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）；Peking University（北京大学）；xAI；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）
