arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.02790cs.LGcs.CL
收藏

利用大语言模型的真实深度

Leveraging the true depth of LLMs

University of Geneva(日内瓦大学);EPFL(瑞士联邦理工学院);FAIR at META(META 的 FAIR)

Ramón Calvo González, Daniele Paliotta, Matteo Pagliardini, Martin Jaggi, François Fleuret

AI总结:

通过重构计算图并并行评估连续层对,提升大语言模型的推理效率,同时保持较高的准确率。

摘要:

大语言模型(LLMs)的出色能力被其巨大的计算成本所掩盖。尽管近期的研究表明,许多LLM层可以重新排列甚至删除,对准确性影响很小,但这些见解尚未转化为显著的推理加速。为了弥合这一差距,我们提出了一种新的方法,通过并行评估连续层对来重新构建计算图。该方法无需重新训练,在Llama 2 7B上实现了1.19倍的吞吐量提升,同时仅将平均基准准确率降低了1.5%。我们展示了该方法在大规模LLM部署中的实际价值,并证明通过轻量级微调并行化层可以恢复部分丢失的准确性。

英文摘要:

The remarkable capabilities of Large Language Models (LLMs) are overshadowed by their immense computational cost. While recent work has shown that many LLM layers can be reordered or even removed with minimal impact on accuracy, these insights have not been translated into significant inference speedups. To bridge this gap, we introduce a novel method that restructures the computational graph by grouping and evaluating consecutive layer pairs in parallel. This approach, requiring no retraining, yields a 1.19x throughput gain on Llama 2 7B while reducing the average benchmark accuracy by only 1.5\%. We demonstrate the practical value of this method for large-scale LLM deployment and show that some of the lost accuracy can be recovered with lightweight fine-tuning of the parallelized layers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑