面向大语言模型的高效推理：投机思维链
Efficient Reasoning for LLMs through Speculative Chain-of-Thought
AI总结：
针对大型推理模型的高延迟问题，本文提出投机思维链（SCoT），通过大小模型协作进行思维级草拟与选择，显著降低推理延迟并保持接近目标模型的性能。
摘要：
大型推理语言模型（如 OpenAI-o1 和 Deepseek-R1）因其出色的任务解决能力近来受到广泛关注。然而，巨大的模型规模以及冗长思维链的生成带来了显著的推理成本和响应延迟。现有的高效推理方法主要侧重于减少模型参数数量或缩短思维链长度。在本文中，我们提出投机思维链（SCoT），通过大小模型协作加速平均推理速度，从另一角度降低推理延迟。SCoT 使用轻量级草稿模型进行思维级草拟，随后选择最佳的思维链草稿，并用目标模型纠正错误案例。所提出的思维行为对齐提高了草拟效率，而草稿选择策略则维持了目标模型在复杂任务上的预测准确性。在 GSM8K、MATH、GaoKao、CollegeMath 和 Olympiad 数据集上的实验结果表明，SCoT 在 Deepseek-R1-Distill-Qwen-32B 和 Deepseek-R1-Distill-Llama-70B 上分别将推理延迟降低了 48%～66% 和 21%～49%，同时实现了接近目标模型级别的性能。我们的代码可在 https://github.com/Jikai0Wang/Speculative_CoT 获取。
英文摘要：
Large reasoning language models such as OpenAI-o1 and Deepseek-R1 have recently attracted widespread attention due to their impressive task-solving abilities. However, the enormous model size and the generation of lengthy thought chains introduce significant reasoning costs and response latency. Existing methods for efficient reasoning mainly focus on reducing the number of model parameters or shortening the chain-of-thought length. In this paper, we introduce Speculative Chain-of-Thought (SCoT), which reduces reasoning latency from another perspective by accelerated average reasoning speed through large and small model collaboration. SCoT conducts thought-level drafting using a lightweight draft model. Then it selects the best CoT draft and corrects the error cases with the target model. The proposed thinking behavior alignment improves the efficiency of drafting and the draft selection strategy maintains the prediction accuracy of the target model for complex tasks. Experimental results on GSM8K, MATH, GaoKao, CollegeMath and Olympiad datasets show that SCoT reduces reasoning latency by 48\%$\sim$66\% and 21\%$\sim$49\% for Deepseek-R1-Distill-Qwen-32B and Deepseek-R1-Distill-Llama-70B while achieving near-target-model-level performance. Our code is available at https://github.com/Jikai0Wang/Speculative_CoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
