arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.19306cs.AIcs.CL
收藏

SETS:利用自我验证和自我校正以提高测试时间扩展

SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling

Google Cloud AI Research(谷歌云人工智能研究)

Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, Jinsung Yoon, Sercan Ö Arık

AI总结:

SETS通过结合并行与顺序技术,利用LLMs的自我改进能力,在无需模型训练的情况下提升测试时间扩展性能。

摘要:

近年来,大型语言模型(LLMs)的进步为通过利用测试时间计算来增强复杂推理任务的性能创造了新机会。然而,现有扩展方法存在关键局限:并行方法如重复采样通常效率低下且很快饱和,而顺序方法如SELF-REFINE在几次轮次后难以进一步改进。尽管结合这些方法显示出潜力,但当前方法需要微调的奖励和修订模型。本文提出了Self-Enhanced Test-Time Scaling(SETS),一种简单而有效的方法,通过战略性地结合并行和顺序技术,并充分利用LLMs的自我改进能力,克服了这些限制。SETS利用LLMs固有的自我验证和自我校正能力,在单一框架内统一采样、验证和校正。这使得在无需模型训练的情况下,能够高效且可扩展地进行测试时间计算,从而在复杂任务上实现性能提升。我们在涵盖规划、推理、数学和编程等挑战性基准上的全面实验结果表明,SETS在性能提升和测试时间扩展行为上均优于其他方法。

英文摘要:

Recent advancements in Large Language Models (LLMs) have created new opportunities to enhance performance on complex reasoning tasks by leveraging test-time computation. However, existing scaling methods have key limitations: parallel methods like repeated sampling are often inefficient and quickly saturate, while sequential methods like SELF-REFINE struggle to improve after a few rounds. Although combining these approaches shows promise, current methods require fine-tuned reward and revision models. This paper proposes Self-Enhanced Test-Time Scaling (SETS), a simple yet effective approach that overcomes these limitations by strategically combining parallel and sequential techniques and fully leveraging LLMs' self-improvement abilities. SETS exploits the inherent self-verification and self-correction capabilities of LLMs, unifying sampling, verification, and correction within a single framework. This facilitates efficient and scalable test-time computation for enhanced performance on complex tasks without any model training. Our comprehensive experimental results on challenging benchmarks spanning planning, reasoning, math, and coding demonstrate that SETS achieves significant performance improvements and more advantageous test-time scaling behavior than the alternatives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑