在推理时间计算(ITC)方法中,如Best-of-N和Tree-of-Thoughts,旨在生成高质量且多样的输出候选,但其高温度采样常无法实现有意义的输出多样性。此外,现有ITC方法对推理过程的控制有限,限制了其可解释性。我们提出了STATe Of Thoughts(STATe),一种可解释的ITC方法,通过搜索高层推理模式。STATe用离散且可解释的文本干预取代随机采样:控制器选择编码高层推理选择的动作;生成器根据这些选择生成推理步骤;评估器评分候选物以引导搜索。这种结构化方法带来了三大优势:首先,动作引导的文本干预可靠地影响LLM生成,并产生比温度基采样更大的响应多样性。其次,在论证生成案例研究中,STATe的显式动作序列捕捉了可解释特征,这些特征高度预测输出质量。第三,估计性能与动作选择之间的关联,使我们能够识别有前途但未探索的动作空间区域,并引导生成向这些区域发展。这些结果确立了STATe作为多样且可控文本生成的实用框架,以及理解驱动性能的推理模式的工具。
英文摘要:
Inference-Time-Compute (ITC) methods like Best-of-$n$ and Tree-of-Thoughts are meant to produce output candidates that are both high-quality and diverse, but their use of high-temperature sampling often fails to achieve meaningful output diversity. Moreover, existing ITC methods offer limited control over $\textit{how}$ to perform reasoning, which in turn limits their interpretability. We present $\textbf{STATe-of-Thoughts}$ (STATe), an interpretable ITC method that $\textit{searches}$ over high-level reasoning patterns. STATe branches over discrete and interpretable textual interventions rather than over token-level samples: a $\textit{controller}$ selects actions encoding high-level reasoning choices; a $\textit{generator}$ produces reasoning steps conditioned on those choices; and an $\textit{evaluator}$ scores candidates to guide search. This structured approach yields three main advantages. First, action-guided textual interventions reliably influence LLM generations and produce greater response diversity than temperature-based sampling. Second, in a case study on argument generation, STATe's explicit action sequences capture interpretable features that are highly predictive of output quality. Third, estimating the association between performance and action choices allows us to identify promising yet unexplored regions of the action space and steer generation toward them. STATe is most useful when a task admits multiple solutions and when understanding $\textit{why}$ an output succeeds matters beyond $\textit{whether}$ the output succeeds. Together, these results establish STATe as both a practical framework for diverse and controllable text generation, and as a tool for understanding the reasoning patterns that drive performance.
TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学);Princeton University(普林斯顿大学);Shanghai AI Lab(上海人工智能实验室);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心)
SoT: Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education(大数据与人工智能交通 key laboratory(北京交通大学));School of Computer Science and Technology, Beijing Jiaotong University(计算机科学与技术学院(北京交通大学));University of Montreal(蒙特利尔大学)
Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Technical University of Darmstadt(德累斯顿技术大学);University of Kassel(卡塞尔大学);TU Dortmund University(Dortmund技术大学);TU Center for Trustworthy Data Science and Security, University Alliance Ruhr(TU可信数据科学与安全中心,鲁尔大学联盟);Hessian Center for AI (Hessian.AI)(黑森人工智能中心(黑森.AI));Centre for Cognitive Science, Technical University of Darmstadt(认知科学中心,德累斯顿技术大学)
Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations
Seoul National University(首尔国立大学);Human-Computer Interaction Institute, Carnegie Mellon University(人机交互研究所,卡内基梅隆大学);Language Technologies Institute, Carnegie Mellon University(语言技术研究所,卡内基梅隆大学)
Who Sees What? Structured Thought-Action Sequences for Epistemic Reasoning in LLMs
University of Milan-Bicocca(米兰-比科卡大学);University of Essex(埃塞克斯大学);University of Naples Federico II(那不勒斯费德里科二世大学);TUM School of Social Sciences and Technology(技术与社会科学学院)
Chaining Simultaneous Thoughts for Numerical Reasoning
Tsinghua University(清华大学);Institute for Artificial Intelligence(人工智能研究院);State Key Lab of Intelligent Technology and Systems(智能技术与系统国家重点实验室);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心)