通过奖励引导缝合实现扩散语言模型的测试时扩展
Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching
AI总结：
通过奖励引导缝合实现扩散语言模型的测试时扩展，提升数学和编程任务的准确率并降低延迟
摘要：
利用大语言模型进行推理通常受益于生成多个思路链，但现有聚合策略通常是轨迹级的（例如，选择最佳轨迹或在最终答案上投票），丢弃了部分或
英文摘要：
Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing aggregation strategies are typically trajectory-level (e.g., selecting the best trace or voting on the final answer), discarding useful intermediate work from partial or "nearly correct" attempts. We propose Stitching Noisy Diffusion Thoughts, a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates. Given a problem, we (i) sample many diverse, low-cost reasoning trajectories using a masked diffusion language model, (ii) score every intermediate step with an off-the-shelf process reward model (PRM), and (iii) stitch these highest-quality steps across trajectories into a composite rationale. This rationale is then used to recompute only the final answer. This modular pipeline separates exploration (diffusion) from evaluation and solution synthesis, avoiding monolithic unified hybrids while preserving broad search. Across math reasoning benchmarks, we find that step-level recombination is most beneficial on harder problems, and ablations highlight the importance of the final solver in converting stitched but imperfect rationales into accurate answers. Using low-confidence diffusion sampling with parallel, independent rollouts, our training-free framework improves average accuracy by up to 23.8% across six math and coding tasks. At the same time, it achieves up to a 1.8x latency reduction relative to both traditional diffusion models (e.g., Dream, LLaDA) and unified architectures (e.g., TiDAR). The code is available https://github.com/roymiles/diffusion-stitching
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
