Mathematical reasoning represents a critical frontier in advancing large language models (LLMs). While step-by-step approaches have emerged as the dominant paradigm for mathematical problem-solving in LLMs, the quality of reasoning steps in training data fundamentally constrains the performance of the models. Recent studies have demonstrated that more detailed intermediate steps can enhance model performance, yet existing methods for step expansion either require more powerful external models or incur substantial computational costs. In this paper, we introduce MathFimer, a novel framework for mathematical reasoning step expansion inspired by the ''Fill-in-the-middle'' task from code reasoning. By decomposing solution chains into prefix-suffix pairs and training models to reconstruct missing intermediate steps, we develop a specialized model, MathFimer-7B, on our carefully curated NuminaMath-FIM dataset. We then apply these models to enhance existing mathematical reasoning datasets by inserting detailed intermediate steps into their solution chains, creating MathFimer-expanded versions. Through comprehensive experiments on multiple mathematical reasoning datasets, including MathInstruct, MetaMathQA and etc., we demonstrate that models trained on MathFimer-expanded data consistently outperform their counterparts trained on original data across various benchmarks such as GSM8K and MATH. Our approach offers a practical, scalable solution for enhancing mathematical reasoning capabilities in LLMs without relying on powerful external models or expensive inference procedures.
"Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing
AI Institute, University of South Carolina（南卡罗来纳大学AI研究院）；Indian Institute of Technology, Kharagpur（印度理工学院克勒格布尔分校）；Islamic University of Technology（伊斯兰科技大学）；Stanford University（斯坦福大学）；Amazon AI（亚马逊AI研究院）
Comparative Analysis of Large Language Models for Context-Aware Code Completion using SAFIM Framework
University of California San Diego（加利福尼亚大学圣迭戈分校）；Simon Fraser University（西蒙菲莎大学）；Duke University（杜克大学）；University of Amsterdam（阿姆斯特丹大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning
USTC（University of Science and Technology of China）；Alibaba Group（阿里巴巴集团）；CASIA（Chinese Academy of Sciences Institute of Automation）；SIAT（Institute of Automation, Chinese Academy of Sciences）
CodeV: Empowering LLMs with HDL Generation through Multi-Level Summarization
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所处理器重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Cambricon Technologies（寒武科技）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；University of Science and Technology of China（中国科学技术大学）；Shanghai Innovation Center for Processor Technologies（上海处理器技术创新中心）
