Recent studies have demonstrated that test-time compute scaling effectively improves the performance of small language models (sLMs). However, prior research has mainly examined test-time compute scaling with an additional larger model as a verifier, leaving verification by sLMs underexplored. In this work, we investigate whether sLMs can reliably verify the output candidates under test-time scaling. We find that even with knowledge distillation from larger verifiers, sLMs struggle with verification tasks requiring memorization, such as numerical calculations and fact-checking. To address this limitation, we propose Tool-integrated verification (T1), a two-stage framework that first filters candidates with external tools and then uses an sLM for final verification, offloading memorization-heavy steps to tools such as a code interpreter. Within T1, we prove that offloading to external tools reduces the memorization burden on sLMs and improves test-time scaling performance. Experiments on the MATH benchmark demonstrate that, with T1, a Llama-3.2 1B model under test-time scaling outperforms the significantly larger Llama-3.1 8B model. Moreover, T1 improves the verification accuracy of both process reward models (PRMs) and critic models. Our findings highlight the potential of tool integration to substantially improve the verification abilities of sLMs.
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information
School of Computer Science and Engineering, Central South University(中南大学计算机科学与工程学院);Research Center for SCIR, Harbin Institute of Technology(哈尔滨工业大学SCIR研究中心);Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));Yunnan University(云南大学)
ASTER: Agentic Scaling with Tool-integrated Extended Reasoning
National Key Laboratory for Novel Software Technology, Nanjing University, China(新型软件技术国家实验室);School of Artificial Intelligence, Nanjing University, China(人工智能学院);Department of Foundation Model, 2012 Labs, Huawei(基础模型部门);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))