发表机构
University of Chicago; Capital One（芝加哥大学; 第一资本金融公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对大语言模型长推理痕迹的问题，引入结构化思维框架，通过构建数据集微调模型，使其采用结构化推理风格提升性能，还能实现上下文修剪，在数学任务中可节省内存并保持一定性能。
AI 中文摘要
大型语言模型擅长生成长思维链，但长推理痕迹往往冗长且内存效率低。本文引入结构化思维框架，将推理组织成交替的<尝试>和<结果>块，<尝试>记录探索性草稿，<结果>包含该步骤的精炼结论。通过将推理痕迹分割成<尝试>块并让语言模型总结成<结果>来构建数据集。在重新格式化的数据上微调预训练基础模型，能使模型采用结构化推理风格，在推理基准测试中性能提升高达8.08%。显式结构还能实现上下文修剪，概念验证修剪实现平均85%的内存/上下文节省，数学任务性能下降8.67%。
英文摘要
Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternating <try> and <outcome> blocks: <try> captures exploratory scratch work, while <outcome> contains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces into <try> blocks and prompting an LLM to summarize each step into its corresponding <outcome>. Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08\% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each <try>/<outcome> pair, the <try> can be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85\% memory / context savings with an 8.67\% performance drop across mathematical tasks.
