With the widespread use of language models (LMs) in NLP tasks, researchers have discovered the potential of Chain-of-thought (CoT) to assist LMs in accomplishing complex reasoning tasks by generating intermediate steps. However, human thought processes are often non-linear, rather than simply sequential chains of thoughts. Therefore, we propose Graph-of-Thought (GoT) reasoning, which models human thought processes not only as a chain but also as a graph. By representing thought units as nodes and connections between them as edges, our approach captures the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. GoT adopts a two-stage framework with an additional GoT encoder for thought graph representation and fuses the graph representation with the original input representation through a gated fusion mechanism. We evaluate GoT's performance on a text-only reasoning task (AQUA-RAT) and a multimodal reasoning task (ScienceQA). Our model achieves significant improvement over the strong CoT baseline on the AQUA-RAT test set and boosts accuracy from 85.19% to 87.59% using the T5-base model over the state-of-the-art Multimodal-CoT on the ScienceQA test set.
Thought Graph: Generating Thought Process for Biological Reasoning
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Arizona State University（亚利桑那州立大学）；University of Pennsylvania（宾夕法尼亚大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；California Institute of Technology（加利福尼亚理工学院）
Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments
School of Information Science and Technology, VISTEC（信息科学与技术学院，VISTEC）；KAIST（韩国科学技术院）；Cohere；SCB 10X；AI Singapore（AI新加坡）；Department of Computer Engineering, Chulalongkorn University（朱拉隆梭大学计算机工程系）
Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorer
Clemson University（克莱姆森大学）；Université de Montréal（蒙特利尔大学）；Center for Advanced AI, Accenture（Accenture高级人工智能中心）；The University of Alabama（阿拉巴马大学）；University of Kansas（堪萨斯大学）；Arizona State University（亚利桑那州立大学）
A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settings
Beijing University of Posts and Telecommunications（北京邮电大学）；Dept. of Comp. Sci. & Tech., Tsinghua University（清华大学计算机科学与技术系）；Northeastern University（东北大学）；Institute for AI, Tsinghua University（清华大学人工智能研究院）；Beijing National Research Center for Information Science and Technology（北京信息科学研究中心）
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step
University of California Los Angeles（加利福尼亚大学洛杉矶分校）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of Southern California（南加利福尼亚大学）；Yonsei University（延世大学）；University of Washington（华盛顿大学）
ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Information, Renmin University of China（中国人民大学信息学院）；Beijing Key Laboratory of Big Data Management and Analysis Methods（北京大数据管理与分析方法重点实验室）
Center for Data Science, New York University（纽约大学数据科学中心）
Large language models (LLMs) have shown remarkable reasoning capabilities given chain-of-thought prompts (examples with intermediate reasoning steps). Existing benchmarks measure reasoning ability indirectly, by evaluating accuracy on downstream tasks such as mathematical reasoning. However, it is unclear how these models obtain the answers and whether they rely on simple heuristics rather than the generated chain-of-thought. To enable systematic exploration of the reasoning ability of LLMs, we present a new synthetic question-answering dataset called PrOntoQA, where each example is generated …
RATT: A Thought Structure for Coherent and Correct LLM Reasoning
Portland State University（波特兰州立大学）；Renmin University of China（中国人民大学）；Pennsylvania State University（宾夕法尼亚州立大学）；Dalian Maritime University（大连海事大学）；University of Kansas（堪萨斯大学）
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
Department of Computing, The Hong Kong Polytechnic University（计算机系，香港理工大学）；Ningbo Digital Twin Institute, Eastern Institute of Technology（宁波数字孪生研究院，东部技术研究所）
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. Th…
Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to i…
Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoning
Tencent BAC（腾讯BAC）；Shenzhen International Graduate School, Tsinghua University（深圳国际研究生院，清华大学）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；School of Mathematics and Statistics, University of Glasgow（格拉斯哥大学数学与统计学学院）
ThoughtSource: A central hub for large language model reasoning data
Medical University of Vienna（维也纳医科大学）；Technical University of Denmark（丹麦技术大学）；Copenhagen University Hospital（哥本哈根大学医院）；The Royal Institute of Technology (KTH)（皇家理工学院（KTH））
Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）人工智能方向）；Didichuxing Co. Ltd（滴滴出行有限公司）；CSE, The Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程系）
