Post-training reasoning LLMs often requires making a tradeoff between costly supervision and compute inefficiency: supervised finetuning typically requires reasoning traces, which is data inefficient since in many applications one only has prompt-answer tuples and lacks the groundtruth chain-of-thought (CoT) sequence. On the other hand, reinforcement learning (with verifiable rewards, for example) can operate without CoT data but it is much more compute-inefficient since it requires multiple autoregressive completions per optimization step, none of which are guaranteed to give a non-zero policy gradient. In this work we explore a middleground, a self-distillation technique that uses answer-only data, treating CoT as latent variables. This relies on training a partner ``inverse reasoning LLM" that generates the original's reasoning traces conditioned on the answer. The inverse model is trained once and reused across incoming batches of answer-only data, with all generation happening offline and up front, making the approach particularly suited to continual post-training. Using Qwen3-0.6B on the Countdown and GSM8K tasks, we show that after GRPO training has saturated, a single offline distillation round from the inverse model roughly doubles pass@1 on hard prompts and, on Countdown, outperforms rejection-sampling self-distillation on every hard-tail metric at every mix ratio (with smaller gains on GSM8K); rejection sampling by construction has no signal on problems the model never solves on its own. Continued GRPO on the fresh data retains an edge at hard pass@1 but trails distillation at pass@8 and pass@32 while regenerating rollouts at every step. As an added bonus we see that the inverse model has a second application as an accurate verifier of correctness. When used for best-of-$n$ selection at test time, it raises selection accuracy on Countdown from 0.753 (majority voting) to 0.969.
Standardization of Post-Publication Code Verification is Possible with the Support of the Community
Department of Mathematics, Pusan National University, Busan, South Korea(釜山国立大学数学系);Industrial Mathematics Center, Pusan National University, Busan, South Korea(釜山国立大学工业数学中心);Humanoid Olfactory Display Center, Pusan National University, Yangsan, Gyeongsangnam-do, South Korea(釜山国立大学人形嗅觉显示中心);Department of Mathematics, Yonsei University, Seoul, South Korea(延世大学数学系);Institute for Future Earth, Pusan National University, Busan, South Korea(釜山国立大学未来地球研究所)
Forward-Backward Reasoning in Large Language Models for Mathematical Verification
Southern University of Science and Technology(南方科技大学);Hong Kong University of Science and Technology(香港科技大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Peking University(北京大学)
Voting for the right answer: Adversarial defense for speaker verification
Tsinghua-CUHK Joint Research Center for Media Sciences, Technologies and Systems(清华-香港中文大学媒体科学、技术与系统联合研究中心);Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Graduate Institute of Communication Engineering, National Taiwan University(台湾大学通信工程研究所);Center for Speech and Language Technologies, Tsinghua University(清华大学语音与语言技术中心)
The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学);University of Cambridge(剑桥大学);University of Edinburgh(爱丁堡大学);City University of Hong Kong(香港城市大学)
Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond
Columbia University(哥伦比亚大学);Department of Computer Science, New York University(纽约大学计算机科学系);Microsoft Research(微软研究院);Center for Data Science, New York University(纽约大学数据科学中心)