Large language models (LLMs) have demonstrated remarkable capabilities in problem-solving. However, their proficiency in solving mathematical problems remains inadequate. We propose MathScale, a simple and scalable method to create high-quality mathematical reasoning data using frontier LLMs (e.g., {\tt GPT-3.5}). Inspired by the cognitive mechanism in human mathematical learning, it first extracts topics and knowledge points from seed math questions and then build a concept graph, which is subsequently used to generate new math questions. MathScale exhibits effective scalability along the size axis of the math dataset that we generate. As a result, we create a mathematical reasoning dataset (MathScaleQA) containing two million math question-answer pairs. To evaluate mathematical reasoning abilities of LLMs comprehensively, we construct {\sc MwpBench}, a benchmark of Math Word Problems, which is a collection of ten datasets (including GSM8K and MATH) covering K-12, college, and competition level math problems. We apply MathScaleQA to fine-tune open-source LLMs (e.g., LLaMA-2 and Mistral), resulting in significantly improved capabilities in mathematical reasoning. Evaluated on {\sc MwpBench}, MathScale-7B achieves state-of-the-art performance across all datasets, surpassing its best peers of equivalent size by 42.9\% in micro average accuracy and 43.7\% in macro average accuracy, respectively.
MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学北京校区人工智能学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE(教育部下一代智能搜索与推荐工程技术研究中心);School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院)
Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
University of Cambridge(剑桥大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Mila, University of Montreal(蒙特利尔大学Mila研究所);Google DeepMind(谷歌DeepMind)
PARAMANU-GANITA: Can Small Math Language Models Rival with Large Language Models on Mathematical Reasoning?
Gyan AI Research(智研AI研究院);CNRS(法国国家科学研究中心);LIG(信息与信号处理实验室);Grenoble INP(格勒诺布尔理工学院);Univ. Grenoble Alpes(格勒诺布尔大学);Indian Institute of Technology Kanpur(坎普尔印度理工学院)
LLMs can be easily Confused by Instructional Distractions
IPAI, Seoul National University(IPAI,首尔国立大学);LG AI Research(LG人工智能研究所);Dept. of ECE, Seoul National University(电子工程系,首尔国立大学);SNU-LG AI Research Center(SNU-LG人工智能研究所)
Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning
DataArc Tech Ltd.(DataArc科技有限公司);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));IDEA Research, International Digital Economy Academy(IDEA研究所,国际数字经济学院);Hithink RoyalFlush Information Network Co., Ltd(Hithink RoyalFlush信息网络有限公司)
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Microsoft Corporation(微软公司);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
WizardMath 通过 RLEIF 方法在不依赖外部 Python 工具的情况下提升大语言模型的数学推理能力,在 GSM8k 和 MATH 基准上超越多个顶级开源及商用模型。
Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction
Beijing University of Posts and Telecommunications(北京邮电大学);AI Lab of Lenovo Research(联想研究院AI实验室);Tianjin University(天津大学);The Hong Kong University of Science and Technology(香港科技大学)
Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical Reasoning
The Hong Kong Polytechnic University(香港理工大学);Research Centre for Data Science & Artificial Intelligence(数据科学与人工智能研究中心);The Chinese University of Hong Kong(香港中文大学)
BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks
Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems, Zhejiang University(浙江无障碍感知与智能系统重点实验室,浙江大学);Alibaba Group(阿里巴巴集团);Worcester Polytechnic Institute(沃斯特理工学院);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and DataSecurity(杭州高新技术区(滨江)区块链与数据安全研究院)
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
Microsoft(微软);Singapore University of Technology and Design(新加坡科技设计大学);Peking University(北京大学);Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学)
Mathematical Computation and Reasoning Errors by Large Language Models
Institute for Intelligent Systems(智能系统研究所);University of Memphis(密苏里大学);AI4STEM Education Center(AI4STEM教育中心);University of Georgia(佐治亚大学);Educational Testing Service(教育测量服务)
IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
Intrinsic Lab, Centre for Sensors, Instrumentation and Cyber-Physical System Engineering (Centre for SeNSE), Indian Institute of Technology Delhi(印度理工学院德里分校传感器、仪器仪表和网络物理系统工程中心(传感中心)本征实验室);RSL Quantum, FITT, IIT Delhi(印度理工学院德里分校 FITT 学院 RSL 量子实验室)
Riazi-8B: An Urdu Large Language Model for Mathematical Reasoning
School of Electrical Engineering and Computer Science (SEECS)(电气工程与计算机科学学院);National University of Sciences and Technology (NUST)(国家科学与技术大学);Department of Communication, Quality Management and Information Systems(通信、质量管理与信息系统系);Mid Sweden University(中瑞典大学)
School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);Lab of Artificial Intelligence for Education, East China Normal University(华东师范大学教育人工智能实验室)
PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
Technical University of Munich(慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Munich Data Science Institute(慕尼黑数据科学研究所);Applied AI Institute(应用人工智能研究所);Charles University(查尔斯大学);Nazarbayev University(纳扎尔拜大学);Inria(法国国家信息与自动化研究所);Indian Institute of Technology, Kharagpur (IIT Kharagpur)(印度Kharagpur理工学院);German University of Digital Science(德国数字科学大学)