arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.04718cs.CLcs.AI
收藏

T1:小语言模型测试时计算扩展的工具集成验证

T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models

KAIST(韩国科学技术院);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);KRAFTON

Minki Kang, Jongwon Jeong, Jaewoong Cho

AI总结:

针对小语言模型在测试时扩展中验证能力不足的问题,提出T1框架,通过外部工具过滤候选输出后由小语言模型进行最终验证,显著提升验证准确率和测试时扩展性能。

摘要:

近期研究表明,测试时计算扩展能有效提升小语言模型(sLMs)的性能。然而,先前研究主要利用额外的大模型作为验证器进行测试时计算扩展,而sLMs自身的验证能力尚未被充分探索。本文研究sLMs在测试时扩展中能否可靠地验证输出候选。我们发现,即使从大验证器进行知识蒸馏,sLMs在需要记忆的任务(如数值计算和事实核查)上仍表现不佳。为解决这一局限,我们提出工具集成验证(T1),这是一个两阶段框架:首先用外部工具过滤候选,然后使用sLM进行最终验证,将记忆密集型步骤卸载到代码解释器等工具上。在T1框架内,我们证明卸载到外部工具可减轻sLMs的记忆负担,并提升测试时扩展性能。在MATH基准上的实验表明,采用T1的Llama-3.2 1B模型在测试时扩展下性能优于规模更大的Llama-3.1 8B模型。此外,T1提高了过程奖励模型(PRMs)和评论家模型的验证准确率。我们的发现凸显了工具集成在显著提升sLMs验证能力方面的潜力。

英文摘要:

Recent studies have demonstrated that test-time compute scaling effectively improves the performance of small language models (sLMs). However, prior research has mainly examined test-time compute scaling with an additional larger model as a verifier, leaving verification by sLMs underexplored. In this work, we investigate whether sLMs can reliably verify the output candidates under test-time scaling. We find that even with knowledge distillation from larger verifiers, sLMs struggle with verification tasks requiring memorization, such as numerical calculations and fact-checking. To address this limitation, we propose Tool-integrated verification (T1), a two-stage framework that first filters candidates with external tools and then uses an sLM for final verification, offloading memorization-heavy steps to tools such as a code interpreter. Within T1, we prove that offloading to external tools reduces the memorization burden on sLMs and improves test-time scaling performance. Experiments on the MATH benchmark demonstrate that, with T1, a Llama-3.2 1B model under test-time scaling outperforms the significantly larger Llama-3.1 8B model. Moreover, T1 improves the verification accuracy of both process reward models (PRMs) and critic models. Our findings highlight the potential of tool integration to substantially improve the verification abilities of sLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑