从隐式到显式：面向大语言模型数学推理的高效令牌逻辑监督
From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs
AI总结：
本文提出FSLR框架，通过显式监督提升大语言模型对逻辑关系的理解能力，实验显示其在推理准确率和训练效率上均优于CoT-SFT。
摘要：
近期研究表明，大语言模型在数学问题解决中表现出有限的逻辑推理能力，通常依赖模式匹配和记忆。我们系统分析了这一限制，聚焦于逻辑关系理解，发现与该能力相关的错误占错误预测的90%以上，而CoT-SFT未能显著减少这些错误。为解决这一瓶颈，我们提出First-Step Logical Reasoning (FSLR)，一种轻量级训练框架，旨在提升逻辑关系理解。我们的关键发现是，第一步规划（确定使用哪些变量和操作）促使模型直接从问题陈述中推导出逻辑关系。通过在该步骤上训练模型，FSLR为逻辑关系理解提供显式监督，不同于CoT-SFT将此类关系隐式嵌入完整解决方案轨迹中。在多个模型和数据集上的广泛实验表明，FSLR在分布内和分布外设置下均优于CoT-SFT，平均提升分别为3.2%和4.6%。此外，FSLR的训练速度提高4-6倍，训练令牌消耗减少超过80%。
英文摘要：
Recent studies reveal that large language models (LLMs) exhibit limited logical reasoning abilities in mathematical problem-solving, instead often relying on pattern-matching and memorization. We systematically analyze this limitation, focusing on logical relationship understanding, which is a core capability underlying genuine logical reasoning, and reveal that errors related to this capability account for over 90\% of incorrect predictions, with Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) failing to substantially reduce these errors. To address this bottleneck, we propose First-Step Logical Reasoning (FSLR), a lightweight training framework targeting logical relationship understanding. Our key insight is that the first planning step-identifying which variables to use and which operation to apply-encourages the model to derive logical relationships directly from the problem statement. By training models on this isolated step, FSLR provides explicit supervision for logical relationship understanding, unlike CoT-SFT which implicitly embeds such relationships within complete solution trajectories. Extensive experiments across multiple models and datasets demonstrate that FSLR consistently outperforms CoT-SFT under both in-distribution and out-of-distribution settings, with average improvements of 3.2\% and 4.6\%, respectively. Moreover, FSLR achieves 4-6x faster training and reduces training token consumption by over 80\%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
