Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);School of Computer Science, Carnegie Mellon University(卡内基梅隆大学计算机学院)
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, Yiming Yang
While the scaling laws of large language models (LLMs) training have been extensively studied, optimal inference configurations of LLMs remain underexplored. We study inference scaling laws (aka test-time scaling laws) and compute-optimal inference, focusing on the trade-offs between model sizes and generating additional tokens with different inference strategies. As a first step towards understanding and designing compute-optimal inference methods, we studied cost-performance trade-offs for inference strategies such as greedy search, majority voting, best-of-$n$, weighted voting, and two different tree search algorithms, using different model sizes and compute budgets. Our findings suggest that scaling inference compute with inference strategies can be more computationally efficient than scaling model parameters. Additionally, smaller models combined with advanced inference algorithms offer Pareto-optimal trade-offs in cost and performance. For example, the Llemma-7B model, when paired with our novel tree search algorithm, consistently outperforms the Llemma-34B model across all tested inference strategies on the MATH benchmark. We hope these insights contribute to a deeper understanding of inference scaling laws (test-time scaling laws) for LLMs.
Beijing International Center for Mathematical Research, Peking University(北京大学北京国际数学研究中心);Center for Quantitative Biology, Center for Machine Learning Research, Peking University(北京大学定量生物学中心、机器学习研究中心);Academy for Multidisciplinary Studies, Capital Normal University(首都师范大学跨学科研究院);Beijing National Center for Applied Mathematics(北京应用数学中心)