TSLM：用于发散思维的树状语言建模
TSLM: Tree-Structured Language Modeling for Divergent Thinking
AI总结：
TSLM通过树状结构语言建模实现发散思维，利用完整搜索树训练提升推理效率和鲁棒性。
摘要：
语言模型按顺序生成推理，阻止在搜索过程中解耦无关的探索路径。我们引入了树状语言建模（TSLM），使用特殊标记来编码分支结构，使模型能够在单次生成过程中生成并选择性地扩展多个搜索路径。通过在包含成功和失败尝试的完整搜索树上训练，TSLM学会内化系统性探索，而无需重复计算共享前缀。TSLM通过避免外部搜索方法所需的多次独立前向传递，实现了稳健的性能和优越的推理效率。这些结果表明了一种新的推理时间扩展范式，证明了在语言模型中通过监督学习完整树状轨迹可以提供一种高效的方法来开发系统性探索能力。
英文摘要：
Language models generate reasoning sequentially, preventing them from decoupling irrelevant exploration paths during search. We introduce Tree-Structured Language Modeling (TSLM), which uses special tokens to encode branching structure, enabling models to generate and selectively expand multiple search paths within a single generation process. By training on complete search trees including both successful and failed attempts, TSLM learns to internalize systematic exploration without redundant recomputation of shared prefixes. TSLM achieves robust performance and superior inference efficiency by avoiding the multiple independent forward passes required by external search methods. These results suggest a new paradigm of inference-time scaling for robust reasoning, demonstrating that supervised learning on complete tree-structured traces provides an efficient alternative for developing systematic exploration capabilities in language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
