通过丰富指令微调实现系统2数学推理
System-2 Mathematical Reasoning via Enriched Instruction Tuning
AI总结：
针对LLMs多步数学推理数据稀缺问题，提出EIT方法，通过ERP和ERS结合人类与AI反馈生成细粒度推理轨迹微调模型，无需符号验证即在GSM8K和MATH上超越现有方法。
摘要：
通过系统2推理解决复杂数学问题是人类的一项自然技能，然而它对当前大型语言模型（LLMs）仍然是一个重大挑战。我们发现深思熟虑的多步推理数据的稀缺性是主要的限制因素。为此，我们引入了丰富指令微调（EIT），该方法通过协同人类和AI的反馈来丰富现有的人类标注数学数据集，以创建细粒度的推理轨迹。这些数据集随后被用于微调开源LLMs，在无需依赖任何符号验证程序的情况下增强其数学推理能力。具体而言，EIT由两个关键步骤组成：推理计划丰富（ERP）和推理步骤丰富（ERS）。前者生成一个高层计划，将复杂指令分解为一系列更简单的目标，而ERS填补人类标注者经常忽略的推理上下文，为LLM微调创建更平滑的推理轨迹。与仅依赖LLM内部知识生成推理链的现有CoT提示方法不同，我们的方法利用人类标注的初始答案作为“元知识”，帮助LLMs生成更详细和精确的推理过程，从而为复杂数学问题带来更可信赖的LLM专家。在实验中，EIT在GSM8K上达到84.1%的准确率，在MATH上达到32.5%，超越了最先进的微调和提示方法，甚至匹敌了工具增强方法的性能。
英文摘要：
Solving complex mathematical problems via system-2 reasoning is a natural human skill, yet it remains a significant challenge for current large language models (LLMs). We identify the scarcity of deliberate multi-step reasoning data as a primary limiting factor. To this end, we introduce Enriched Instruction Tuning (EIT), a method that enriches existing human-annotated mathematical datasets by synergizing human and AI feedback to create fine-grained reasoning trajectories. These datasets are then used to fine-tune open-source LLMs, enhancing their mathematical reasoning abilities without reliance on any symbolic verification program. Concretely, EIT is composed of two critical steps: Enriching with Reasoning Plan (ERP) and Enriching with Reasoning Step (ERS). The former generates a high-level plan that breaks down complex instructions into a sequence of simpler objectives, while ERS fills in reasoning contexts often overlooked by human annotators, creating a smoother reasoning trajectory for LLM fine-tuning. Unlike existing CoT prompting methods that generate reasoning chains only depending on LLM's internal knowledge, our method leverages human-annotated initial answers as ``meta-knowledge'' to help LLMs generate more detailed and precise reasoning processes, leading to a more trustworthy LLM expert for complex mathematical problems. In experiments, EIT achieves an accuracy of 84.1% on GSM8K and 32.5% on MATH, surpassing state-of-the-art fine-tuning and prompting methods, and even matching the performance of tool-augmented methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
