Large Language Models (LLMs) have shown excellent performance in language understanding, text generation, code synthesis, and many other tasks, while they still struggle in complex multi-step reasoning problems, such as mathematical reasoning. In this paper, through a newly proposed arithmetical puzzle problem, we show that the model can perform well on multi-step reasoning tasks via fine-tuning on high-quality synthetic data. Experimental results with the open-llama-3B model on three different test datasets show that not only the model can reach a zero-shot pass@1 at 0.44 on the in-domain dataset, it also demonstrates certain generalization capabilities on the out-of-domain datasets. Specifically, this paper has designed two out-of-domain datasets in the form of extending the numerical range and the composing components of the arithmetical puzzle problem separately. The fine-tuned models have shown encouraging performance on these two far more difficult tasks with the zero-shot pass@1 at 0.33 and 0.35, respectively.
Atomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities
Sun Yat-sen University（中山大学）；Tsinghua University（清华大学）；ByteDance Inc（字节跳动公司）；Arizona State University（亚利桑那州立大学）；Tencent Youtu Lab（腾讯优图实验室）；Fudan University（复旦大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology（广东省消防科学与智能应急技术重点实验室）；University of Illinois Chicago（伊利诺伊大学香槟分校）
Exploring Context Window of Large Language Models via Decomposed Positional Vectors
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Baichuan Inc.（百川公司）；School of Information, Renmin University of China（中国人民大学信息学院）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）
The Hong Kong Polytechnic University（香港理工大学）；Southern University of Science and Technology（南方科技大学）；Tsinghua University（清华大学）；Reallm Labs（Reallm实验室）；Wuhan University（武汉大学）；The University of Hong Kong（香港大学）
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Shanghai Key Laboratory of Intelligent Information Processing, Fudan University（复旦大学上海智能信息处理重点实验室）
Exploring Communication Strategies for Collaborative LLM Agents in Mathematical Problem-Solving
Institute for Intelligent Systems, University of Memphis, Memphis, TN, USA（智能系统研究所，密苏里大学）；Department of Electrical and Computer Engineering, University of Memphis, Memphis, TN, USA（电气与计算机工程系，密苏里大学）；AI4STEM Education Center, University of Georgia, Athens, GA, USA（AI4STEM教育中心，佐治亚大学）；National GENIUS Center, Athens, GA, USA（国家GENIUS中心）；Department of Mathematics, Science, and Social Studies Education, University of Georgia, Athens, GA, USA（数学、科学与社会科学教育系，佐治亚大学）；Faculty of Education, The University of Hong Kong, Hong Kong, PR China（教育学院，香港大学）；Educational Testing Service, Princeton, NJ, USA（教育测评服务）；Department of Applied Social Sciences, Hong Kong Polytechnic University, Hong Kong, PR China（应用社会科学系，香港理工大学）；Department of Psychology, University of Memphis, Memphis, TN, USA（心理学系，密苏里大学）
该研究使用 GPT-4o 在 MATH 数据集上比较双智能体数学解题的四种沟通模式，发现同伴协作准确率最高，并表明有效对话行为是提升 AI 教育协作解题的关键。
Exploring Task Performance with Interpretable Models via Sparse Auto-Encoders
The University of Sheffield（谢菲尔德大学）；Tianjin University（天津大学）；University of Manchester（曼彻斯特大学）；Kyushu University（九州大学）；City University of Hong Kong（香港城市大学）
