Seer Self-Consistency:面向自适应测试时缩放的先进预算估计
Seer Self-Consistency: Advance Budget Estimation for Adaptive Test-Time Scaling
AI总结:
SeerSC通过整合系统1和系统2推理,提升自适应测试时缩放的令牌效率和延迟,实现47%的令牌消耗减少和43%的推理延迟降低。
摘要:
测试时缩放提高了大型语言模型(LLMs)的推理性能,但也带来了显著的计算成本。尽管最近的研究通过动态自一致性减少了令牌消耗,但它们仍然受到顺序请求高延迟的限制。在本文中,我们提出了SeerSC,一种动态自一致性框架,通过整合系统1和系统2推理同时提高令牌效率和延迟。具体而言,我们利用快速的系统1计算给定查询的答案熵。这个分数随后用于评估样本的扩展潜力,从而在系统2下实现动态自一致性。得益于系统1提供的先进和准确的估计,所提出的方法可以减少令牌使用,同时通过并行生成实现显著的延迟降低。它优于现有方法,实现了高达47%的令牌消耗减少和43%的推理延迟降低,而不会显著损失性能。
英文摘要:
Test-time scaling improves the inference performance of Large Language Models (LLMs) but also incurs substantial computational costs. Although recent studies have reduced token consumption through dynamic self-consistency, they remain constrained by the high latency of sequential requests. In this paper, we propose SeerSC, a dynamic self-consistency framework that simultaneously improves token efficiency and latency by integrating System 1 and System 2 reasoning. Specifically, we utilize the rapid System 1 to compute the answer entropy for given queries. This score is then used to evaluate the potential of samples for scaling, enabling dynamic self-consistency under System 2. Benefiting from the advance and accurate estimation provided by System 1, the proposed method can reduce token usage while simultaneously achieving a significant decrease in latency through parallel generation. It outperforms existing methods, achieving up to a 47% reduction in token consumption and a 43% reduction in inference latency without significant performance loss.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。