arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.00724cs.AI
收藏

推理缩放定律:关于语言模型问题求解的计算最优推理的实证分析

Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models

Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);School of Computer Science, Carnegie Mellon University(卡内基梅隆大学计算机学院)

Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, Yiming Yang

AI总结:

本文研究大语言模型的推理缩放定律与计算最优推理,发现较小模型结合先进推理算法(如新型树搜索)在成本与性能上可实现帕累托最优,比单纯扩展模型参数更高效。

摘要:

尽管大语言模型(LLMs)训练的缩放定律已被广泛研究,但LLMs的最优推理配置仍未被充分探索。我们研究了推理缩放定律(即测试时缩放定律)和计算最优推理,重点关注模型大小与使用不同推理策略生成额外token之间的权衡。作为理解和设计计算最优推理方法的第一步,我们使用不同的模型大小和计算预算,研究了多种推理策略的成本-性能权衡,包括贪心搜索、多数投票、best-of-$n$、加权投票以及两种不同的树搜索算法。我们的发现表明,使用推理策略扩展推理计算比扩展模型参数在计算上更高效。此外,较小的模型结合先进的推理算法在成本和性能上提供了帕累托最优权衡。例如,Llemma-7B模型与我们的新型树搜索算法相结合,在MATH基准的所有测试推理策略上均持续优于Llemma-34B模型。我们希望这些见解能促进对LLMs推理缩放定律(测试时缩放定律)的更深入理解。

英文摘要:

While the scaling laws of large language models (LLMs) training have been extensively studied, optimal inference configurations of LLMs remain underexplored. We study inference scaling laws (aka test-time scaling laws) and compute-optimal inference, focusing on the trade-offs between model sizes and generating additional tokens with different inference strategies. As a first step towards understanding and designing compute-optimal inference methods, we studied cost-performance trade-offs for inference strategies such as greedy search, majority voting, best-of-$n$, weighted voting, and two different tree search algorithms, using different model sizes and compute budgets. Our findings suggest that scaling inference compute with inference strategies can be more computationally efficient than scaling model parameters. Additionally, smaller models combined with advanced inference algorithms offer Pareto-optimal trade-offs in cost and performance. For example, the Llemma-7B model, when paired with our novel tree search algorithm, consistently outperforms the Llemma-34B model across all tested inference strategies on the MATH benchmark. We hope these insights contribute to a deeper understanding of inference scaling laws (test-time scaling laws) for LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑