ClozeMath：通过学习填充方程提升语言模型的数学推理能力
ClozeMath: Improving Mathematical Reasoning in Language Models by Learning to Fill Equations
AI总结：
本文提出 ClozeMath 方法，通过让语言模型学习填充被遮蔽方程的完形填空式任务来提升数学推理能力，实验表明其在多个数学基准上优于 Masked Thought，并具有更好的鲁棒性。
摘要：
大型语言模型（LLMs）的能力已通过在与人类思维过程相符的数据上进行训练而得到增强，例如思维链（Chain-of-Thought）格式。然而，有证据表明，传统的下一词预测方案可能无法完全捕捉人类学习思考的方式。受人类泛化数学推理方式的启发，我们提出了一种名为 ClozeMath 的新方法，用于对大型语言模型进行数学推理微调。我们的 ClozeMath 包含一个文本填充任务，该任务从给定解答中预测被遮蔽的方程，类似于人类学习中使用的完形填空练习。在 GSM8K、MATH 和 GSM-Symbolic 上的实验表明，在使用两种测试时缩放解码算法——束搜索（Beam Search）和思维链解码（Chain-of-Thought decoding）时，ClozeMath 在性能和鲁棒性上均超越了强基线方法 Masked Thought。此外，我们还进行了消融研究，以分析各种架构和实现选择对我们方法的影响。
英文摘要：
The capabilities of large language models (LLMs) have been enhanced by training on data that reflects human thought processes, such as the Chain-of-Thought format. However, evidence suggests that the conventional scheme of next-word prediction may not fully capture how humans learn to think. Inspired by how humans generalize mathematical reasoning, we propose a new approach named ClozeMath to fine-tune LLMs for mathematical reasoning. Our ClozeMath involves a text-infilling task that predicts masked equations from a given solution, analogous to cloze exercises used in human learning. Experiments on GSM8K, MATH, and GSM-Symbolic show that ClozeMath surpasses the strong baseline Masked Thought in performance and robustness, with two test-time scaling decoding algorithms, Beam Search and Chain-of-Thought decoding. Additionally, we conduct an ablation study to analyze the effects of various architectural and implementation choices on our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
