SIaM：大型语言模型的自我改进代码辅助数学推理
SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models
AI总结：
本文提出SIaM范式，利用基于代码的评论家模型指导数据构建、质量控制与评估，并结合自生成数据对齐算法，有效提升了LLMs的代码辅助数学推理能力及泛化表现。
摘要：
通过编程教导大型语言模型（LLMs）解决数学问题的趋势日益增长。现有研究主要侧重于提示强大的闭源模型生成种子训练数据，随后进行领域内数据增强，从而赋予LLMs相当强的代码辅助数学推理能力。然而，持续在源自GSM8K等少数数据集的增强数据上训练这些模型，可能会损害其泛化能力，并使其有效性局限于狭窄的问题类型范围内。相反，通过利用大规模、专家编写、多样化的数学问答对来改进此类LLMs的潜力仍未被探索。为了利用这些资源并应对代码响应评估等独特挑战，我们提出了一种新范式，该范式使用基于代码的评论家模型来指导问题-代码数据构建、质量控制和补充评估等步骤。我们还探索了使用自生成指令/偏好数据的不同对齐算法，以促进持续改进。在英文和中文的领域内（高达+5.7%）和领域外（+4.4%）基准上的实验证明了所提出范式的有效性。
英文摘要：
There is a growing trend of teaching large language models (LLMs) to solve mathematical problems through coding. Existing studies primarily focus on prompting powerful, closed-source models to generate seed training data followed by in-domain data augmentation, equipping LLMs with considerable capabilities for code-aided mathematical reasoning. However, continually training these models on augmented data derived from a few datasets such as GSM8K may impair their generalization abilities and restrict their effectiveness to a narrow range of question types. Conversely, the potential of improving such LLMs by leveraging large-scale, expert-written, diverse math question-answer pairs remains unexplored. To utilize these resources and tackle unique challenges such as code response assessment, we propose a novel paradigm that uses a code-based critic model to guide steps including question-code data construction, quality control, and complementary evaluation. We also explore different alignment algorithms with self-generated instruction/preference data to foster continuous improvement. Experiments across both in-domain (up to +5.7%) and out-of-domain (+4.4%) benchmarks in English and Chinese demonstrate the effectiveness of the proposed paradigm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
