Large Language Models (LLMs) have limited performance when solving arithmetic reasoning tasks and often provide incorrect answers. Unlike natural language understanding, math problems typically have a single correct answer, making the task of generating accurate solutions more challenging for LLMs. To the best of our knowledge, we are not aware of any LLMs that indicate their level of confidence in their responses which fuels a trust deficit in these models impeding their adoption. To address this deficiency, we propose `MathPrompter', a technique that improves performance of LLMs on arithmetic problems along with increased reliance in the predictions. MathPrompter uses the Zero-shot chain-of-thought prompting technique to generate multiple Algebraic expressions or Python functions to solve the same math problem in different ways and thereby raise the confidence level in the output results. This is in contrast to other prompt based CoT methods, where there is no check on the validity of the intermediate steps followed. Our technique improves over state-of-the-art on the MultiArith dataset ($78.7\%\rightarrow92.5\%$) evaluated using 175B parameter GPT-based LLM.
Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
organization= School of Electrical Engineering;Computer Science, National University of Science;organization= School of Computing, Data;Mathematical Sciences, Western Sydney University, Indonesia;organization= Department of Communication, Quality Management;Information Systems, Mid Sweden University, Östersund Campus, Sweden
The Hong Kong Polytechnic University(香港理工大学);Southern University of Science and Technology(南方科技大学);Tsinghua University(清华大学);Reallm Labs(Reallm实验室);Wuhan University(武汉大学);The University of Hong Kong(香港大学)
A Survey on Large Language Models for Mathematical Reasoning
National Key Laboratory for Novel Software Technology(国家新型软件技术重点实验室);School of Artificial Intelligence(人工智能学院);Nanyang Technological University(南洋理工大学);Skywork AI Singapore(Skywork AI新加坡);The Chinese University of Hong Kong Shenzhen(香港中文大学(深圳))
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
The Hong Kong University of Science and Technology(香港科学与技术大学);Huawei Noah’s Ark Lab(华为诺亚实验室);Zhejiang University(浙江大学);Peking University(北京大学);The Hong Kong Polytechnic University(香港理工大学);University of Surrey(萨里大学)
Distilling Mathematical Reasoning Capabilities into Small Language Models
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions
The Hong Kong Polytechnic University(香港理工大学);University of Electronic Science and Technology of China(电子科技大学);Tencent Youtu Lab(腾讯优图实验室);Beihang University(北京航空航天大学);Simon Fraser University(西蒙弗雷泽大学)
Forward-Backward Reasoning in Large Language Models for Mathematical Verification
Southern University of Science and Technology(南方科技大学);Hong Kong University of Science and Technology(香港科技大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Peking University(北京大学)
An Investigation of Robustness of LLMs in Mathematical Reasoning: Benchmarking with Mathematically-Equivalent Transformation of Advanced Mathematical Problems
Department of Computer Science University of Illinois Urbana–Champaign(计算机科学系伊利诺伊大学厄巴纳-香槟分校);Department of Computer Science Stanford University(计算机科学系斯坦福大学)
Towards Spoken Mathematical Reasoning: Benchmarking Speech-based Models over Multi-faceted Math Problems
Institute for Infocomm Research (I 2 R), A*STAR, Singapore(信息与通信研究 institute(I2R),A*STAR,新加坡);Centre for Frontier AI Research (CFAR), A*STAR, Singapore(前沿人工智能研究 centre(CFAR),A*STAR,新加坡)
Riazi-8B: An Urdu Large Language Model for Mathematical Reasoning
School of Electrical Engineering and Computer Science (SEECS)(电气工程与计算机科学学院);National University of Sciences and Technology (NUST)(国家科学与技术大学);Department of Communication, Quality Management and Information Systems(通信、质量管理与信息系统系);Mid Sweden University(中瑞典大学)