SEED：通过调度投机解码加速推理树构建
SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding
AI总结：
本文提出SeeD框架，通过调度投机解码和轮次调度策略，加速推理树构建，同时优化运行速度和GPU内存管理，在三个推理数据集上实现显著加速。
摘要：
大型语言模型（LLMs）在各种任务中展现出显著的涌现能力，但在复杂推理和规划任务上仍显不足。基于树搜索的推理方法通过超越思维链提示的能力，鼓励探索中间步骤，从而解决了这一问题。然而，这些方法由于系统性地探索和评估多条思维路径，引入了显著的推理延迟。本文介绍了SeeD，一种新颖且高效的推理框架，旨在同时优化运行速度和GPU内存管理。通过采用调度投机执行，SeeD高效地处理思维生成和状态评估的多次迭代，利用轮次调度策略来管理草稿模型的调度。在三个推理数据集上的广泛实验评估表明，SeeD具有优越的加速性能，为免训练的投机解码中的批处理推理提供了一条可行路径。
英文摘要：
Large Language Models (LLMs) demonstrate remarkable emergent abilities across various tasks, yet fall short of complex reasoning and planning tasks. The tree-search-based reasoning methods address this by surpassing the capabilities of chain-of-thought prompting, encouraging exploration of intermediate steps. However, such methods introduce significant inference latency due to the systematic exploration and evaluation of multiple thought paths. This paper introduces SeeD, a novel and efficient inference framework to optimize runtime speed and GPU memory management concurrently. By employing a scheduled speculative execution, SeeD efficiently handles multiple iterations for the thought generation and the state evaluation, leveraging a rounds-scheduled strategy to manage draft model dispatching. Extensive experimental evaluations on three reasoning datasets demonstrate superior speedup performance of SeeD, providing a viable path for batched inference in training-free speculative decoding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
