R2-Write：基于深度推理的开放性写作反思与修订
R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoning
AI总结：
本文提出R2-Write框架，通过迭代作家-评判者交互生成高质量思考轨迹，增强开放性写作的深度推理能力，实验表明显式引入反思与修订模式显著提升了写作任务性能。
摘要：
尽管长链推理显著提升了大型语言模型在可验证领域如数学中的表现，但其在开放性任务如写作中的有效性尚不明确。本文系统研究发现，现有主流推理模型在开放性写作任务中收益有限。进一步分析表明，这些模型缺乏开放性写作中的深度反思与修订模式，导致改进幅度远小于数学推理任务。为解决此问题，我们引入R2-Write：一个自动化框架，通过迭代作家-评判者交互生成富含显式反思与修订模式的高质量思考轨迹。为防止冗余反思，我们设计了一种过程奖励机制，监督强化学习中的反思质量，提升性能和令牌效率。在多个创意写作和深度研究基准上的广泛实验表明，显著改进，验证了显式引入反思与修订模式解锁了开放性写作任务的深度推理能力。
英文摘要：
While deep reasoning with long chain-of-thought has dramatically improved large language models in verifiable domains like mathematics, its effectiveness for open-ended tasks such as writing remains unexplored. In this paper, we conduct a systematic investigation revealing that existing mainstream reasoning models achieve limited gains on open-ended writing tasks. Our further analysis shows that these models lack deep reflection and revision patterns in open-ended writing, resulting in substantially smaller improvements compared to mathematical reasoning tasks. To address this limitation, we introduce R2-Write: an automated framework that synthesizes high-quality thinking trajectories enriched with explicit reflection and revision patterns through iterative writer-judge interaction. To prevent redundant reflections, we design a process reward mechanism that supervises reflection quality during reinforcement learning, improving both performance and token efficiency. Extensive experiments across multiple creative writing and deep-research benchmarks demonstrate significant improvements, validating that explicitly incorporating reflection and revision patterns unlocks deep reasoning capabilities for open-ended writing tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
