Reinforcement learning (RL) for mathematical reasoning can suffer from reward sparsity: for challenging problems, LLM fails to sample any correct trajectories, preventing RL from receiving meaningful positive feedback. At the same time, there often exist human-written reference solutions along with the problem (e.g., problems from AoPS), but directly fine-tuning on these solutions offers no benefit because models often cannot imitate human proofs that lie outside their own reasoning distribution.
We introduce Reference-Guided Fine-Tuning (ReGFT), a simple and effective method that utilizes human-written reference solutions to synthesize positive trajectories on hard problems and train on them before RL. For each problem, we provide the model with a partial reference solution and let it generate its own reasoning trace, ensuring the resulting trajectories remain in the model's reasoning space while still benefiting from reference guidance.
Fine-tuning on these reference-guided trajectories increases the number of solvable problems and produces a checkpoint that receives more positive rewards during RL. Across three benchmarks (AIME24, AIME25, BeyondAIME), ReGFT consistently improves supervised accuracy, accelerates DAPO training, and raises the final performance plateau of RL. Our results show that ReGFT effectively overcomes reward sparsity and unlocks stronger RL-based mathematical reasoning.
RefTool: Reference-Guided Tool Creation for Knowledge-Intensive Reasoning
Wangxuan Institute of Computer Technology, Peking University(北京大学王萱计算机技术研究所);University of Chicago(芝加哥大学);University of California, Los Angeles(加州大学洛杉矶分校)
Transferable Mass Spectrum Prediction via Reference-Guided Test-time Specialization
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The University of Hong Kong(香港大学);South China University of Technology(华南理工大学);Hong Kong Polytechnic University(香港理工大学);Jinan University(暨南大学);The Hong Kong University of Science and Technology(香港科技大学)
REGEN: Reference-Guided Synthetic Multivariate Time Series Generation for Forecasting
Birla AI Labs, Office of Ananya Birla(Birla AI实验室,Ananya Birla办公室);Birla Institute of Technology and Science, Pilani(Birla理工学院与科学学院,Pilani);Kalinga Institute of Industrial Technology, Bhubaneswar(Kalinga工业技术学院,Bhubaneswar)
Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
Peng Cheng Laboratory(鹏城实验室);The University of Tokyo(东京大学);Southern University of Science and Technology(南方科技大学);Peking University(北京大学);Malardalen University(马拉达伦大学);LocationMind Inc.(LocationMind公司)
Finite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows
The University of Tokyo(东京大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);Nanyang Technological University(南洋理工大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能研究中心)