本文提出Caterpillar of Thoughts算法，通过理论分析证明最优算法生成caterpillar树，从而减少生成次数并提升性能。
摘要：
大型语言模型（LLMs）在允许使用额外测试时间计算时，如采样、思维链、回溯或修订部分解，通常能生成更优的输出。尽管这些技术在经验上取得成功，但对其推理时间计算结构的理论理解有限。本文将测试时间计算建模为与马尔可夫链交互的算法：在任意时刻，算法可以从任何之前观察到的状态继续生成。许多现有测试时间算法，如思维链（CoT）、思维树（ToT）或最佳k（Best-of-k）都可以视为该模型中的特定算法。我们证明，虽然回溯可以指数级减少生成次数，但非常有限形式的回溯在理论上是足够的。即我们证明最优算法始终生成caterpillar树。即如果移除最优算法生成的状态树的叶子，我们得到一条路径。受我们对最优算法的表征启发，我们提出Caterpillar of Thoughts（CaT）新测试时间计算算法，减少token/状态生成次数。我们的实证评估显示，与ToT相比，CaT在提升成功率的同时也减少了token生成次数。
英文摘要：
Large language models (LLMs) can often produce substantially better outputs when allowed to use additional test-time computation, such as sampling, chain of thought, backtracking, or revising partial solutions. Despite the growing empirical success of such techniques, there is limited theoretical understanding of how inference time computation should be structured, or what constitutes an optimal use of a fixed computation budget.
We model test-time computation as an algorithm interacting with a Markov chain: at any point, the algorithm may resume generation from any previously observed state. That is, unlike standard Markov chains where the states are drawn passively, we allow the algorithm to backtrack to any previously observed state of the Markov chain at any time. Many of the existing test-time algorithms, such as Chain-of-Thought (CoT) (Wei et al., 2023), Tree-of-Thoughts (ToT) (Yao et al., 2023), or Best-of-$k$ (Brown et al., 2024) could be seen as specific algorithms in this model.
We prove that while backtracking can reduce the number of generations exponentially, a very limited form of backtracking is theoretically sufficient. Namely, we show that the optimal algorithm always generates a caterpillar tree. That is, if we remove the leaves of the state tree generated by the optimal algorithm, we obtain a path. Motivated by our characterization of the optimal algorithm, we present Caterpillar of Thoughts (CaT), a new test-time computation algorithm, reducing the number of token/state generations. Our empirical evaluation shows that CaT, compared to ToT, achieves a better success rate while also reducing the number of token generations.
Reasoning aligns language models to human cognition
ETH AI Center, Zürich, Switzerland；ETH Zürich, Switzerland；Institute for Neuroinformatics, University of Zürich \& ETH Zürich, Switzerland；University College London, London, United Kingdom
An Asymptotic Theory of Chain-of-Thought in In-Context Learning
Department of Physics, Graduate School of Science, The University of Tokyo（东京大学物理系研究生院）；John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保罗森工程与应用科学学院）；Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University（哈佛大学凯普勒人工智能研究 institute）；Center for Brain Science, Harvard University（哈佛大学脑科学中心）
Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；University College London（伦敦大学学院）；Yale University（耶鲁大学）；The Hong Kong University of Science and Technology（香港科技大学）；University of Washington（华盛顿大学）
提出Uncertainty of Thoughts (UoT)算法，通过不确定性感知模拟、基于不确定性的奖励和奖励传播机制，增强大型语言模型主动提问寻求信息的能力，在医疗诊断等任务中平均成功率提升38.1%。
