DeepSeekMath:突破开放语言模型在数学推理上的极限
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
AI总结:
本文提出 DeepSeekMath 7B 模型,通过精选网络数学数据进行预训练,并引入群体相对策略优化(GRPO)算法,在竞赛级 MATH 基准上接近 GPT-4 性能,显著提升了开放语言模型的数学推理能力。
摘要:
数学推理因其复杂且高度结构化的特性,对语言模型构成了重大挑战。本文提出 DeepSeekMath 7B,它基于 DeepSeek-Coder-Base-v1.5 7B 继续预训练,使用了从 Common Crawl 获取的 1200 亿与数学相关的 token,并辅以自然语言和代码数据。在不依赖外部工具包和投票技术的情况下,DeepSeekMath 7B 在竞赛级 MATH 基准上取得了 51.7% 的优异成绩,接近 Gemini-Ultra 和 GPT-4 的性能水平。对来自 DeepSeekMath 7B 的 64 个样本进行自洽性测试,在 MATH 上达到 60.9%。DeepSeekMath 的数学推理能力归功于两个关键因素:首先,通过精心设计的数据选择流程,我们充分利用了公开网络数据的巨大潜力。其次,我们提出近端策略优化(PPO)的变体——群体相对策略优化(GRPO),在增强数学推理能力的同时,优化了 PPO 的内存占用。
英文摘要:
Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature. In this paper, we introduce DeepSeekMath 7B, which continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data. DeepSeekMath 7B has achieved an impressive score of 51.7% on the competition-level MATH benchmark without relying on external toolkits and voting techniques, approaching the performance level of Gemini-Ultra and GPT-4. Self-consistency over 64 samples from DeepSeekMath 7B achieves 60.9% on MATH. The mathematical reasoning capability of DeepSeekMath is attributed to two key factors: First, we harness the significant potential of publicly available web data through a meticulously engineered data selection pipeline. Second, we introduce Group Relative Policy Optimization (GRPO), a variant of Proximal Policy Optimization (PPO), that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。