发表机构
Nanjing University（南京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出将科学论文展开为多轮生成轨迹的流程构建CPT语料库，经CPT后微调可提升写作性能与长文档阅读能力，混合SFT数据可进一步优化学术写作表现。
AI 中文摘要
近期一类合成数据研究致力于重构现有文本背后的思考过程而非改写文本本身，但这类研究仅针对短网页段落，仅能恢复局部思考，且未触及整篇文档的结构。科学论文具有清晰且高度统一的结构，是将该范式提升至文档级的天然载体。我们提出了一套流程，将每篇论文展开为多轮生成轨迹，其中教师模型重构整篇论文的写作过程：包含写作请求、全局计划以及每个章节的写作前考量。所有章节文本和摘要均保留原论文的原文。我们将该流程应用于经质量筛选的arXiv论文，获得了规模约为源文本两倍的持续预训练（CPT）语料库。相同的反向构建可扩展至指令数据和评估：将真实论文文本作为答案可得到一个SFT数据集；在保留论文中锚定任务可得到PAW-Bench，这是一个学术写作基准，其任务带有自身的评分标准和检查清单。在受控实验中，使用我们的语料库进行CPT后再在公共数据集上进行监督微调，可广泛提升写作基准性能，同时保留通用推理能力并改善长文档阅读能力。即使每个模型都在专用写作SFT数据集上进行微调，这种写作提升仍然存在；将我们的SFT数据混入该流程可进一步提升学术写作表现。
英文摘要
A recent line of synthetic-data work reconstructs the thinking behind existing text rather than rewriting the text itself, but it operates on short web passages, recovers only local thoughts, and leaves the structure of whole documents untouched. Scientific papers are written to a clear and largely uniform structure and make a natural substrate for lifting this paradigm to the document level. We present a pipeline that unfolds each paper into a multi-turn generation trajectory in which a teacher model reconstructs the writing process of the whole paper: a writing request, a global plan, and pre-writing deliberation for each section. All section texts and the abstract are kept verbatim from the source paper. We apply the pipeline to 1.8M quality-filtered arXiv papers and obtain a 60B-token corpus for continued pre-training (CPT) that is roughly twice the source text. The same reverse construction extends to instruction data and evaluation. We build an SFT dataset of 200K samples using answers derived from paper text. We also use held-out papers to construct PAW-Bench, a benchmark of 2,940 academic writing tasks with per-task rubrics and checklists. In controlled experiments, CPT on our corpus followed by SFT on public datasets improves writing performance while preserving general reasoning and improving long document reading. Replacing part of the writing SFT data with our synthetic instruction data further improves performance on PAW-Bench.
