arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.23701cs.CLcs.AI
收藏

现代大语言模型中早期退出解码的边际效益

The Diminishing Returns of Early-Exit Decoding in Modern LLMs

Stevens Institute of Technology(史蒂文斯理工学院);Northeastern University(东北大学);Stony Brook University(石溪大学)

Rui Wei, Rui Du, Hanfei Yu, Devesh Tiwari, Jian Li, Zhaozhuo Xu, Hao Wang

AI总结:

研究现代大语言模型中早期退出解码的有效性,分析中间表示演变,并提出衡量模型内在适合性的指标,发现新模型代际中早期退出效果递减,密集变换器比专家混合和状态空间模型更适合早期退出。

摘要:

在大语言模型(LLM)推理中,早期退出指在预测足够自信时在中间层停止计算,从而降低延迟和成本。然而,最近的LLM采用改进的预训练配方和架构,减少了层冗余,可能限制早期退出机会。我们重新评估现代LLM中的逐层早期退出,并分析训练过程中中间表示的演变。我们引入了一个指标来量化模型的内在适合性,并提出一个基准,供研究人员探索不同模型和工作负载上的潜在早期退出收益。我们的结果表明,在新模型代际中早期退出效果呈递减趋势。此外,密集变换器通常比专家混合和状态空间模型具有更大的早期退出潜力。此外,更大模型,特别是参数超过200亿的模型,以及未经过专门调优的基预训练模型,通常表现出更高的早期退出潜力。

英文摘要:

In Large Language Model (LLM) inference, early-exit refers to stopping computation at an intermediate layer once the prediction is sufficiently confident, thereby reducing latency and cost. However, recent LLMs adopt improved pretraining recipes and architectures that reduce layer redundancy, potentially limiting early-exit opportunities. We re-evaluate layer-wise early-exit in modern LLMs and analyze how intermediate representations evolve during training. We introduce a metric to quantify a model's intrinsic suitability for early-exit and propose a benchmark for researchers to explore the potential early-exit benefits on different models and workloads. Our results show a diminishing trend in early-exit effectiveness across newer model generations. We further find that dense transformers generally offer greater early-exit potential than Mixture-of-Experts and State Space Models. In addition, larger models, particularly those with more than 20 billion parameters, and base pretrained models without specialized tuning tend to exhibit higher early-exit potential.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑