通过推测性搜索加速大型语言模型推理
Accelerating Large Language Model Reasoning via Speculative Search
AI总结：
提出SpecSearch框架，通过小型模型与大型模型在思想和令牌层面协作，并引入保质量拒绝机制，在保持推理质量的同时实现高达2.12倍的加速。
摘要：
基于树搜索的推理方法通过促进对多个中间推理步骤（即思想）的探索，显著增强了大型语言模型（LLM）的推理能力。然而，这些方法因必须生成大量推理思想而遭受显著的推理延迟，严重限制了LLM的适用性。为解决这一挑战，我们提出了一种新颖的推测性搜索（SpecSearch）框架，通过优化思想生成来显著加速LLM推理。具体而言，SpecSearch利用一个小型模型在思想和令牌两个层面上与大型模型进行战略性协作，高效生成高质量的推理思想。SpecSearch的主要支柱是一种新颖的保质量拒绝机制，该机制有效过滤掉质量低于大型模型输出的思想。此外，我们证明SpecSearch保持了与大型模型相当的推理质量。在Qwen和Llama模型上的实验表明，SpecSearch显著优于最先进的方法，在保持相当推理质量的同时实现了高达2.12倍的加速。
英文摘要：
Tree-search-based reasoning methods have significantly enhanced the reasoning capability of large language models (LLMs) by facilitating the exploration of multiple intermediate reasoning steps, i.e., thoughts. However, these methods suffer from substantial inference latency, as they have to generate numerous reasoning thoughts, severely limiting LLM applicability. To address this challenge, we propose a novel Speculative Search (SpecSearch) framework that significantly accelerates LLM reasoning by optimizing thought generation. Specifically, SpecSearch utilizes a small model to strategically collaborate with a large model at both thought and token levels, efficiently generating high-quality reasoning thoughts. The major pillar of SpecSearch is a novel quality-preserving rejection mechanism, which effectively filters out thoughts whose quality falls below that of the large model's outputs. Moreover, we show that SpecSearch preserves comparable reasoning quality to the large model. Experiments on both the Qwen and Llama models demonstrate that SpecSearch significantly outperforms state-of-the-art approaches, achieving up to 2.12$\times$ speedup with comparable reasoning quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
