Test-Time Scaling (TTS) is an important method for improving the performance of Large Language Models (LLMs) by using additional computation during the inference phase. However, current studies do not systematically analyze how policy models, Process Reward Models (PRMs), and problem difficulty influence TTS. This lack of analysis limits the understanding and practical use of TTS methods. In this paper, we focus on two core questions: (1) What is the optimal approach to scale test-time computation across different policy models, PRMs, and problem difficulty levels? (2) To what extent can extended computation improve the performance of LLMs on complex tasks, and can smaller language models outperform larger ones through this approach? Through comprehensive experiments on MATH-500 and challenging AIME24 tasks, we have the following observations: (1) The compute-optimal TTS strategy is highly dependent on the choice of policy model, PRM, and problem difficulty. (2) With our compute-optimal TTS strategy, extremely small policy models can outperform larger models. For example, a 1B LLM can exceed a 405B LLM on MATH-500. Moreover, on both MATH-500 and AIME24, a 0.5B LLM outperforms GPT-4o, a 3B LLM surpasses a 405B LLM, and a 7B LLM beats o1 and DeepSeek-R1, while with higher inference efficiency. These findings show the significance of adapting TTS strategies to the specific characteristics of each task and model and indicate that TTS is a promising approach for enhancing the reasoning abilities of LLMs.
Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
MAIS, Institute of Automation, Chinese Academy of Sciences(自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);University of California, Santa Barbara(加州大学圣巴巴拉分校);Beijing Wenge Technology Co., Ltd(北京文格科技有限公司)
Rethinking the Instruction Quality: LIFT is What You Need
Shanghai Jiao Tong University(上海交通大学);Microsoft Cloud and AI(微软云与人工智能);Jilin University(吉林大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Rethinking Test-Time Scaling for Medical AI: Model and Task-Aware Strategies for LLMs and VLMs
Department of Biomedical Systems Informatics, Yonsei University College of Medicine(生物医学系统信息学系,延世大学医学院);Yonsei University College of Medicine(延世大学医学院);Department of Radiation Oncology, Yonsei University College of Medicine(放射肿瘤学系,延世大学医学院);Department of Biomedical Systems Informatics and the Department of Psychiatry, Yonsei University College of Medicine(生物医学系统信息学系和精神病学系,延世大学医学院);Institute for Innovation in Digital Healthcare, Yonsei University(数字医疗创新研究所,延世大学)
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(人工智能通用智能国家重点实验室,智能科学与技术学院,北京大学);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);School of Economics, Peking University(经济学院,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学)
Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoning
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工程学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖先进研究院技术研究所);Huawei Noah’s Ark Lab(华为诺亚实验室)
Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);RIKEN Center for Advanced Intelligence Project (AIP)(RIKEN先进人工智能项目中心);School of Computer Science and Technology, Qilu University of Technology(齐鲁大学计算机科学与技术学院);The University of Tokyo(东京大学)
Rethinking On-policy Optimization for Query Augmentation
University of Utah(犹他大学);The University of Queensland(昆士兰大学);University of Waterloo(滑铁卢大学);New York University(纽约大学);University of Notre Dame(圣母大学);Université de Montréal(蒙特利尔大学);Google DeepMind(谷歌DeepMind);University of Oklahoma(俄克拉荷马大学)
Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China(中国人民大学人工智能学院);Beijing Key Laboratory of Research on Large Models(北京大型模型研究关键实验室);Engineering Research Center of Next-Generation Intelligent Search(下一代智能搜索工程研究中心);College of Artificial Intelligence, Tianjin University of Science(天津科技大学人工智能学院)