Active Prompting with Chain-of-Thought for Large Language Models
The Hong Kong University of Science and Technology(香港科技大学);University of Toronto(多伦多大学);The University of Hong Kong(香港大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
The increasing scale of large language models (LLMs) brings emergent abilities to various complex tasks requiring reasoning, such as arithmetic and commonsense reasoning. It is known that the effective design of task-specific prompts is critical for LLMs' ability to produce high-quality answers. In particular, an effective approach for complex question-and-answer tasks is example-based prompting with chain-of-thought (CoT) reasoning, which significantly improves the performance of LLMs. However, current CoT methods rely on a fixed set of human-annotated exemplars, which are not necessarily the most effective examples for different tasks. This paper proposes a new method, Active-Prompt, to adapt LLMs to different tasks with task-specific example prompts (annotated with human-designed CoT reasoning). For this purpose, we propose a solution to the key problem of determining which questions are the most important and helpful ones to annotate from a pool of task-specific queries. By borrowing ideas from the related problem of uncertainty-based active learning, we introduce several metrics to characterize the uncertainty so as to select the most uncertain questions for annotation. Experimental results demonstrate the superiority of our proposed method, achieving state-of-the-art on eight complex reasoning tasks. Further analyses of different uncertainty metrics, pool sizes, zero-shot learning, and accuracy-uncertainty relationship demonstrate the effectiveness of our method. Our code will be available at https://github.com/shizhediao/active-prompt.
Zero-Shot Chain-of-Thought Reasoning Guided by Evolutionary Algorithms in Large Language Models
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);National Key Laboratory of General Artificial Intelligence(通用人工智能国家重点实验室);Key Laboratory of Machine Perceptron (MOE), Peking University(北京大学机器感知教育部重点实验室)
Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Wuhan AI Research(武汉人工智能研究院)
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Singapore Management University(新加坡管理大学);Southwest Jiaotong University(西南交通大学);Singapore University of Technology and Design(新加坡科技设计大学);East China Normal University(华东师范大学)
Enhancing Granular Sentiment Classification with Chain-of-Thought Prompting in Large Language Models
John F Kennedy High school Fremont, USA(约翰·F·肯尼迪高中弗里蒙特分校);Queensland Academy for Science Mathematics and Technology(昆士兰科学数学与技术学院);Research Spark Hub Inc(Research Spark Hub公司)
Understanding Before Reasoning: Enhancing Chain-of-Thought with Iterative Summarization Pre-Prompting
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science(电子工程学院,上海工程技术大学);School of Information Science and Engineering, Ningbo University(信息科学与工程学院,宁波大学)
DualCoTs: Dual Chain-of-Thoughts Prompting for Sentiment Lexicon Expansion of Idioms
The College of Big Data and Internet, Shenzhen Technology University(深圳技术大学大数据与互联网学院);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Harbin Institute of Technology(哈尔滨工业大学)
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Alibaba Cloud Computing(阿里巴巴云计算);Shanghai Jiao Tong University(上海交通大学);College of Software, Zhejiang University(浙江大学软件学院)
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Information, Renmin University of China(中国人民大学信息学院);DIRO, Université de Montréal(蒙特利尔大学计算机科学与运筹学系)
SoT: Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education(大数据与人工智能交通 key laboratory(北京交通大学));School of Computer Science and Technology, Beijing Jiaotong University(计算机科学与技术学院(北京交通大学));University of Montreal(蒙特利尔大学)