思维链（Chain-of-Thought, CoT）是一种分步且连贯的推理链，作为大语言模型（large language models, LLM）的提示策略展现出了惊人的优势。近年来，CoT提示的显著效果吸引了越来越多的研究，但目前仍缺乏对CoT提示关键影响因素的系统总结，也缺少提示使用的综合指南。为更深入地理解CoT提示，本文调研了大量现有研究，对可能影响CoT提示效果的若干因素进行了系统全面的分析，并基于这些讨论介绍了如何在不同应用场景中更好地应用该策略。本文进一步分析了CoT提示面临的挑战，提出了若干未来研究方向，本综述可为相关研究提供整体参考。
英文摘要：
Chain-of-Thought (CoT), a step-wise and coherent reasoning chain, shows its impressive strength when used as a prompting strategy for large language models (LLM). Recent years, the prominent effect of CoT prompting has attracted emerging research. However, there still lacks of a systematic summary about key factors of CoT prompting and comprehensive guide for prompts utilizing. For a deeper understanding about CoT prompting, we survey on a wide range of current research, presenting a systematic and comprehensive analysis on several factors that may influence the effect of CoT prompting, and introduce how to better apply it in different applications under these discussions. We further analyze the challenges and propose some future directions about CoT prompting. This survey could provide an overall reference on related research.
Tsinghua University（清华大学）；Microsoft Research Asia（微软亚洲研究院）；Microsoft（微软）
Large language models can perform various reasoning tasks by using chain-of-thought prompting, which guides them to find answers through step-by-step demonstrations. However, the quality of the prompts depends on the demonstrations given to the models, and creating many of them by hand is costly. We introduce Synthetic prompting, a method that leverages a few handcrafted examples to prompt the model to generate more examples by itself, and selects effective demonstrations to elicit better reasoning. Our method alternates between a backward and forward process to generate new examples. The back…
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. Th…
Zero-Shot Chain-of-Thought Reasoning Guided by Evolutionary Algorithms in Large Language Models
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；National Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室）；Key Laboratory of Machine Perceptron (MOE), Peking University（北京大学机器感知教育部重点实验室）
Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Wuhan AI Research（武汉人工智能研究院）
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Singapore Management University（新加坡管理大学）；Southwest Jiaotong University（西南交通大学）；Singapore University of Technology and Design（新加坡科技设计大学）；East China Normal University（华东师范大学）
Enhancing Granular Sentiment Classification with Chain-of-Thought Prompting in Large Language Models
John F Kennedy High school Fremont, USA（约翰·F·肯尼迪高中弗里蒙特分校）；Queensland Academy for Science Mathematics and Technology（昆士兰科学数学与技术学院）；Research Spark Hub Inc（Research Spark Hub公司）
Understanding Before Reasoning: Enhancing Chain-of-Thought with Iterative Summarization Pre-Prompting
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science（电子工程学院，上海工程技术大学）；School of Information Science and Engineering, Ningbo University（信息科学与工程学院，宁波大学）
DualCoTs: Dual Chain-of-Thoughts Prompting for Sentiment Lexicon Expansion of Idioms
The College of Big Data and Internet, Shenzhen Technology University（深圳技术大学大数据与互联网学院）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Harbin Institute of Technology（哈尔滨工业大学）
Chain-of-thought (CoT) prompting enables large language models (LLMs) to solve complex reasoning tasks by generating an explanation before the final prediction. Despite it's promising ability, a critical downside of CoT prompting is that the performance is greatly affected by the factuality of the generated explanation. To improve the correctness of the explanations, fine-tuning language models with explanation data is needed. However, there exists only a few datasets that can be used for such approaches, and no data collection tool for building them. Thus, we introduce CoTEVer, a tool-kit for…
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Alibaba Cloud Computing（阿里巴巴云计算）；Shanghai Jiao Tong University（上海交通大学）；College of Software, Zhejiang University（浙江大学软件学院）
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Information, Renmin University of China（中国人民大学信息学院）；DIRO, Université de Montréal（蒙特利尔大学计算机科学与运筹学系）
While Pre-trained Language Models (PLMs) internalize a great amount of world knowledge, they have been shown incapable of recalling these knowledge to solve tasks requiring complex & multi-step reasoning. Similar to how humans develop a "chain of thought" for these tasks, how can we equip PLMs with such abilities? In this work, we explore an iterative prompting framework, a new prompting paradigm which progressively elicits relevant knowledge from PLMs for multi-step inference. We identify key limitations of existing prompting methods, namely they are either restricted to queries with a single…
Shanghai Jiao Tong University（上海交通大学）；Amazon Web Services（亚马逊云科技）
Large language models (LLMs) can perform complex reasoning by generating intermediate reasoning steps. Providing these steps for prompting demonstrations is called chain-of-thought (CoT) prompting. CoT prompting has two major paradigms. One leverages a simple prompt like "Let's think step by step" to facilitate step-by-step thinking before answering a question. The other uses a few manual demonstrations one by one, each composed of a question and a reasoning chain that leads to an answer. The superior performance of the second paradigm hinges on the hand-crafting of task-specific demonstration…
SoT: Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education（大数据与人工智能交通 key laboratory（北京交通大学））；School of Computer Science and Technology, Beijing Jiaotong University（计算机科学与技术学院（北京交通大学））；University of Montreal（蒙特利尔大学）
