Orca-Math：释放小型语言模型在小学数学中的潜力
Orca-Math: Unlocking the potential of SLMs in Grade School Math
AI总结：
本文提出基于Mistral-7B的70亿参数模型Orca-Math，通过20万道多智能体合成数学题和迭代偏好学习，在GSM8k上以86.81%的pass@1超越多个大模型，无需外部工具或多次调用。
摘要：
数学应用题求解长期以来被认为是小型语言模型（SLMs）的一项复杂任务。最近一项研究推测，要在GSM8K基准上达到超过80%的准确率，所需的最小模型规模为340亿参数。为了用更小的模型达到这一性能水平，研究者通常训练SLM生成Python代码或使用工具来帮助避免计算错误。此外，他们还采用集成方法，将多达100次模型运行的输出结合起来以获得更准确的结果。结果选择通过共识、多数投票或与SLM配合使用的独立验证器模型来完成。集成方法能大幅提升准确率，但代价是多次调用模型带来的显著成本增加（例如，Phi-GSM使用top-48将性能从68.2提升到81.5）。在本文中，我们提出了Orca-Math，一个基于Mistral-7B的70亿参数SLM，它在GSM8k上达到86.81%的准确率，无需多次模型调用，也不使用验证器、代码执行或任何其他外部工具。我们的方法包含以下关键要素：（1）一个包含20万道数学题的高质量合成数据集，通过多智能体协作设置创建，其中智能体协作生成数据；（2）一种迭代学习技术，使SLM能够练习解题、接收对其解答的反馈，并从包含SLM解答和反馈的偏好对中学习。仅使用监督微调训练时，Orca-Math在GSM8k pass@1指标上达到81.50%。通过迭代偏好学习，Orca-Math达到86.81%的pass@1。Orca-Math超越了规模大得多的模型，如LLAMA-2-70B、WizardMath-70B、Gemini-Pro、ChatGPT-3.5。它在使用少得多的数据（数十万道题对比数百万道题）的同时，也显著优于其他更小的模型。
英文摘要：
Mathematical word problem-solving has long been recognized as a complex task for small language models (SLMs). A recent study hypothesized that the smallest model size, needed to achieve over 80% accuracy on the GSM8K benchmark, is 34 billion parameters. To reach this level of performance with smaller models, researcher often train SLMs to generate Python code or use tools to help avoid calculation errors. Additionally, they employ ensembling, where outputs of up to 100 model runs are combined to arrive at a more accurate result. Result selection is done using consensus, majority vote or a separate a verifier model used in conjunction with the SLM. Ensembling provides a substantial boost in accuracy but at a significant cost increase with multiple calls to the model (e.g., Phi-GSM uses top-48 to boost the performance from 68.2 to 81.5). In this work, we present Orca-Math, a 7-billion-parameter SLM based on the Mistral-7B, which achieves 86.81% on GSM8k without the need for multiple model calls or the use of verifiers, code execution or any other external tools. Our approach has the following key elements: (1) A high quality synthetic dataset of 200K math problems created using a multi-agent setup where agents collaborate to create the data, (2) An iterative learning techniques that enables the SLM to practice solving problems, receive feedback on its solutions and learn from preference pairs incorporating the SLM solutions and the feedback. When trained with Supervised Fine-Tuning alone, Orca-Math achieves 81.50% on GSM8k pass@1 metric. With iterative preference learning, Orca-Math achieves 86.81% pass@1. Orca-Math surpasses the performance of significantly larger models such as LLAMA-2-70B, WizardMath-70B, Gemini-Pro, ChatGPT-3.5. It also significantly outperforms other smaller models while using much smaller data (hundreds of thousands vs. millions of problems).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
