Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. In this paper, we study the scaling of inference-time computation in LLMs, with a focus on answering the question: if an LLM is allowed to use a fixed but non-trivial amount of inference-time compute, how much can it improve its performance on a challenging prompt? Answering this question has implications not only on the achievable performance of LLMs, but also on the future of LLM pretraining and how one should tradeoff inference-time and pre-training compute. Despite its importance, little research attempted to understand the scaling behaviors of various test-time inference methods. Moreover, current work largely provides negative results for a number of these strategies. In this work, we analyze two primary mechanisms to scale test-time computation: (1) searching against dense, process-based verifier reward models; and (2) updating the model's distribution over a response adaptively, given the prompt at test time. We find that in both cases, the effectiveness of different approaches to scaling test-time compute critically varies depending on the difficulty of the prompt. This observation motivates applying a "compute-optimal" scaling strategy, which acts to most effectively allocate test-time compute adaptively per prompt. Using this compute-optimal strategy, we can improve the efficiency of test-time compute scaling by more than 4x compared to a best-of-N baseline. Additionally, in a FLOPs-matched evaluation, we find that on problems where a smaller base model attains somewhat non-trivial success rates, test-time compute can be used to outperform a 14x larger model.
Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models
University of California, Berkeley(加州大学伯克利分校);University of Cambridge(剑桥大学);University of Washington(华盛顿大学);University of Toronto(多伦多大学);University of Tokyo(东京大学)
Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis
University of YYY(YYY大学);Institute of WWW(WWW研究所);The Hong Kong University of Science and Technology(香港科技大学);Northwestern Polytechnical University(西北工业大学);University of Science and Technology Beijing(北京科技大学);Hong Kong Baptist University(香港浸会大学);Chinese University of Hong Kong(香港中文大学);Shanghai Mobvoi Information Technology Co., Ltd(上海小问信息技术有限公司);University of Rochester(罗切斯特大学);University of Surrey(萨里大学)
ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning
ML Research, Morgan Stanley, New York, NY 10019, USA(ML研究,摩根大通,纽约,纽约州,10019,美国);Visa Research, Austin, TX 78759, USA(Visa研究,奥斯汀,德克萨斯州,78759,美国);Department of ECE, University of Minnesota, MN 55455, USA(电子工程系,明尼苏达大学,明尼苏达州,55455,美国)
Automated Pattern Detection--An Algorithm for Constructing Optimally Synchronizing Multi-Regular Language Filters
在一维元胞自动机的计算力学结构分析中,出现了以下来自时间序列分析的\emph{change-point problem}的自动机理论模拟:\emph{Given a string $σ$ and a collection $\{\mc{D}_i\}$ of finite automata, identify the regions of $σ$ that belong to each $\mc{D}_i$ and, in particular, the boundaries separating them.}我们提出了两种求解此\emph{multi-regular language filtering problem}的方法。第一种方法虽然提供了理想的解决方案,但需要堆栈,最坏情况下的计算时间以$σ$的长度二次增长,并在任意长的未来输入窗口上的任何点上调节其输出。第二种方法是在算法上构造近似于第一算法的换能器。然而,与基于堆栈的算法相比,传感器只需要有限的内存,在线性时间内运行,并为每个字母读取提供即时输出;此外,它是具有这三个特征的最佳有限状态近似。
ELLIS Institute Tübingen(埃利斯研究所蒂宾根分所);OpenEuroLLM;Eindhoven University of Technology(埃因霍温理工大学);University of Freiburg(弗赖堡大学);Zuse School ELIZA(楚塞ELIZA学校);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Tübingen AI Center(蒂宾根人工智能中心)