注意间隔：为改进链式推理微调而弥合思维跳跃
Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning
AI总结：
本文提出CoT思维跳跃桥任务，通过生成缺失的中间推理步骤提升数学推理任务的性能，实验表明在NuminaMath上改进达+5.87%。
摘要：
大型语言模型（LLMs）通过链式推理（CoT）在数学任务上取得了显著进展。然而，现有的数学Co T数据集往往由于专家省略中间步骤而出现思维跳跃，这会负面影响模型的学习和泛化能力。我们提出了CoT思维跳跃桥任务，旨在自动检测跳跃并生成缺失的中间推理步骤以恢复Co T的完整性和连贯性。为了促进这一点，我们构建了一个专门的训练数据集，称为ScaleQM+，基于结构化的ScaleQuestMath数据集，并训练了CoT-Bridge来弥合思维跳跃。通过在数学推理基准上的全面实验，我们证明了在桥接数据集上微调的模型在原数据集上表现更优，NuminaMath上的改进高达+5.87%。我们的方法有效增强了蒸馏数据（+3.02%）并为强化学习提供了更好的起点（+3.1%），作为插件式模块兼容现有的优化技术。此外，CoT-Bridge在非领域逻辑推理任务中表现出更好的泛化能力，证实了增强推理完整性带来的广泛适用性。
英文摘要：
Large language models (LLMs) have achieved remarkable progress on mathematical tasks through Chain-of-Thought (CoT) reasoning. However, existing mathematical CoT datasets often suffer from Thought Leaps due to experts omitting intermediate steps, which negatively impacts model learning and generalization. We propose the CoT Thought Leap Bridge Task, which aims to automatically detect leaps and generate missing intermediate reasoning steps to restore the completeness and coherence of CoT. To facilitate this, we constructed a specialized training dataset called ScaleQM+, based on the structured ScaleQuestMath dataset, and trained CoT-Bridge to bridge thought leaps. Through comprehensive experiments on mathematical reasoning benchmarks, we demonstrate that models fine-tuned on bridged datasets consistently outperform those trained on original datasets, with improvements of up to +5.87% on NuminaMath. Our approach effectively enhances distilled data (+3.02%) and provides better starting points for reinforcement learning (+3.1%), functioning as a plug-and-play module compatible with existing optimization techniques. Furthermore, CoT-Bridge demonstrate improved generalization to out-of-domain logical reasoning tasks, confirming that enhancing reasoning completeness yields broadly applicable benefits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
