The Process Reward Model (PRM) plays a crucial role in mathematical reasoning tasks, requiring high-quality supervised process data. However, we observe that reasoning steps generated by Large Language Models (LLMs) often fail to exhibit strictly incremental information, leading to redundancy that can hinder effective reasoning. To address this issue, we propose CFPRM, a simple yet effective coarse-to-fine strategy. Instead of focusing on the detection of redundant steps, our approach first establishes a coarse-grained window to merge adjacent reasoning steps into unified, holistic steps. The window size is then progressively reduced to extract fine-grained reasoning steps, enabling data collection at multiple granularities for training. By leveraging this hierarchical refinement process, CFPRM mitigates redundancy while preserving essential fine-grained knowledge. Extensive experiments on two reasoning datasets across three loss criteria validate the CFPRM's effectiveness and versatility.
An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Verifier-Backed Hard Problem Generation for Mathematical Reasoning
Department of Data Science, City University of Hong Kong（香港城市大学数据科学系）；Hong Kong Institute of AI for Science, City University of Hong Kong（香港城市大学人工智能科学研究所）；School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Department of Statistics, University of Oxford（牛津大学统计系）
MathVis-Fine: Aligning Visual Supervision with Necessity via Progressive Dependency-Guided Training for Multimodal Mathematical Reasoning
School of ECE, Peking University（北京大学电子与计算机工程学院）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与技术学院）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；Tencent Youtu Lab（腾讯优图实验室）
OGER: A Robust Offline-Guided Exploration Reward for Hybrid Reinforcement Learning
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学深圳研究院）；Hithink RoyalFlush Information Network, Hangzhou, China（杭州Hithink RoyalFlush信息网络）；Computer and Information Science, University of Macau, Macau, China（澳门大学计算机与信息科学学院）
Cash App Labs（Cash App实验室）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
We convert the DeepMind Mathematics Dataset into a reinforcement learning environment by interpreting it as a program synthesis problem. Each action taken in the environment adds an operator or an input into a discrete compute graph. Graphs which compute correct answers yield positive reward, enabling the optimization of a policy to construct compute graphs conditioned on problem statements. Baseline models are trained using Double DQN on various subsets of problem types, demonstrating the capability to learn to correctly construct graphs despite the challenges of combinatorial explosion and n…
Encouraging Responsible Use of Generative AI in Education: A Reward-Based Learning Approach
Cleveland State University（克利夫兰州立大学）；The Davey Tree Expert Company（戴维树木专家公司）；The MathWorks Inc（迈斯沃克公司）；Youngstown State University（扬斯敦州立大学）；Roux Institute at Northeastern University（东北大学鲁克斯学院）
Fudan University（复旦大学）；Mila（米拉人工智能研究所）；DIRO, Université de Montréal（蒙特利尔大学计算机科学与运筹学系）；CIFAR（加拿大高等研究院）
Math Word Problems (MWP) is an important task that requires the ability of understanding and reasoning over mathematical text. Existing approaches mostly formalize it as a generation task by adopting Seq2Seq or Seq2Tree models to encode an input math problem in natural language as a global representation and generate the output mathematical expression. Such approaches only learn shallow heuristics and fail to capture fine-grained variations in inputs. In this paper, we propose to model a math word problem in a fine-to-coarse manner to capture both the local fine-grained information and the glo…
Mathematical Framework for Custom Reward Functions in Job Application Evaluation using Reinforcement Learning
Department of Computer Science and Engineering, SRM Institute of Science and Technology（计算机科学与工程系，SRM科学与技术学院）；Department of Computer Science and Engineering, Sastra University（计算机科学与工程系，萨斯特拉大学）；Halıcıoğlu Data Science Institute, University of California San Diego（Halıcıoğlu数据科学研究所，加州大学圣地亚哥分校）；O6AI LABS（O6AI实验室）
The Streaming Engine (SE) is a Coarse-Grained Reconfigurable Array which provides programming flexibility and high-performance with energy efficiency. An application program to be executed on the SE is represented as a combination of Synchronous Data Flow (SDF) graphs, where every instruction is represented as a node. Each node needs to be mapped to the right slot and array in the SE to ensure the correct execution of the program. This creates an optimization problem with a vast and sparse search space for which finding a mapping manually is impractical because it requires expertise and knowle…
