优化思维链推理：通过计划增强解决安排瓶颈
Optimizing Chain-of-Thought Reasoning: Tackling Arranging Bottleneck via Plan Augmentation
AI总结：
本文通过将思维链推理分解为安排与执行，发现瓶颈主要在安排环节，并提出基于抽象计划的训练与推理方法，在数学和工具使用任务上显著提升长距离推理泛化能力。
摘要：
大型语言模型的多步推理能力在数学和工具使用等任务中至关重要。当前研究主要集中于通过思维链（CoT）步骤的微调来提升模型在这些多步推理任务上的表现，但这些方法往往具有启发性，既未探索也未解决瓶颈问题。在本研究中，我们将CoT推理细分为两个部分：安排与执行，并发现模型的瓶颈主要在于安排而非执行。基于这一发现，我们提出了一种基于计划的训练与推理方法，引导模型通过抽象计划生成安排步骤。我们在数学（GSM8k）和工具使用（ToolBench）基准上进行了实验。结果表明，与直接使用CoT数据进行微调相比，我们的方法在缓解安排瓶颈方面取得了更好的性能，尤其在长距离推理泛化方面表现出色。
英文摘要：
Multi-step reasoning ability of large language models is crucial in tasks such as math and tool utilization. Current researches predominantly focus on enhancing model performance in these multi-step reasoning tasks through fine-tuning with Chain-of-Thought (CoT) steps, yet these methods tend to be heuristic, without exploring nor resolving the bottleneck. In this study, we subdivide CoT reasoning into two parts: arranging and executing, and identify that the bottleneck of models mainly lies in arranging rather than executing. Based on this finding, we propose a plan-based training and reasoning method that guides models to generate arranging steps through abstract plans. We experiment on both math (GSM8k) and tool utilization (ToolBench) benchmarks. Results show that compared to fine-tuning directly with CoT data, our approach achieves a better performance on alleviating arranging bottleneck, particularly excelling in long-distance reasoning generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
