发表机构
Texas A&M University; Harvard University; Vanderbilt University; Stanford University; DARPA; University of North Carolina at Chapel Hill（德克萨斯A&M大学; 哈佛大学; 范德堡大学; 斯坦福大学; 美国国防高级研究计划局; 北卡罗来纳大学教堂山分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出数学原语概念和\hlei{}基准，诊断LLMs数学推理的四个维度，发现“发现”是主要瓶颈，并引入原语优先的自蒸馏框架\abs{}，有效提升模型数学推理能力。
AI 中文摘要
尽管大语言模型（LLMs）在前沿数学问题上展现了惊人的能力，但它们是否拥有支撑其解决方案的结构性数学理解仍不清楚。本文迈出了系统研究LLMs数学理解的第一步，从诊断其不同能力到利用这些发现改进后训练。首先，我们引入数学原语的概念来探测结构性数学理解，并提出\hlei{}，一个新颖的基准，沿四个不同维度评估数学推理：发现、生成、消化和执行。其次，我们的系统诊断表明，解决方案的准确性掩盖了不同的能力画像，原语解锁了大量的潜在执行能力，而发现是数学推理中的主要瓶颈。我们的后训练分析进一步表明，以发现受限的失败特别容易修复。最后，基于这些发现，我们引入\abs{}，一个原语优先的自蒸馏框架，选择性地将原语引导的推理迁移到学生模型。大量实验表明，\abs{}在不同模型规模和具有挑战性的基准上持续优于基线，改善了数学推理。
英文摘要
While Large Language Models (LLMs) have demonstrated striking capabilities on frontier mathematical problems, it remains unclear whether they possess the structural mathematical understanding underlying their solutions. In this paper, we take a first step toward systematically studying mathematical understanding in LLMs, from diagnosing its distinct capabilities to leveraging these findings to improve post-training. First, we introduce the notion of Mathematical Primitive to probe structural mathematical understanding and propose \hlei{}, a novel benchmark that evaluates mathematical reasoning along four distinct dimensions: Discovery, Generation, Digestion, and Execution. Second, our systematic diagnosis shows that solution accuracy masks distinct capability profiles, primitives unlock substantial latent execution capacity, and Discovery is the dominant bottleneck in mathematical reasoning. Our post-training analysis further shows that discovery-limited failures are particularly amenable to repair. Finally, building on these findings, we introduce \abs{}, a primitive-privileged self-distillation framework that selectively transfers primitive-guided reasoning into the student model. Extensive experiments demonstrate that \abs{} consistently improves mathematical reasoning over baselines across model scales and challenging benchmarks.
Comments27 pages
