戴上思考帽子:从人类推理机制视角对链式思考微调的综述
Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
AI总结:
本文从人类推理机制角度综述了链式思考微调,分析其通过六顶思考帽子框架分类方法,并探讨未来研究方向及现有数据集与模型表现。
摘要:
链式思考(CoT)微调旨在通过训练大型语言模型(LLMs)于精心编纂的推理轨迹上,赋予其推理能力。它利用监督和强化微调来培养LLMs的人类样推理技能,包括详细规划、发散性思维、直觉判断、及时反思、内部思考和事实感知等。随着CoT微调的发展,LLMs在数学推理和代码生成等任务上表现出显著改进。然而,现有关于CoT微调的综述主要关注技术方面,忽略了从人类推理机制视角的系统分析。鉴于CoT微调的最终目标是使LLMs像人类一样推理,有必要通过人类认知的视角来研究这一技术。为此,本文首次全面综述了基于人类推理理论的CoT微调。具体而言,受著名的六顶思考帽子框架启发,该框架系统地用六个隐喻帽子描述常见的思维模式,我们通过这一视角对CoT微调方法进行分类和分析。此外,基于这一理论,我们概述了CoT微调未来研究的潜在方向。此外,我们还编制了现有数据集和模型性能的全面概述,并维护了一个实时的GitHub仓库(https://github.com/AI-Chen/Awesome-CoT-Finetuning),持续跟踪该领域最近的进展。我们希望本文的综述能成为激励创新和促进该快速发展的领域进步的宝贵资源。
英文摘要:
Chain of thought (CoT) fine-tuning aims to endow large language models (LLMs) with reasoning capabilities by training them on curated reasoning traces. It leverages both supervised and reinforced fine-tuning to cultivate human-like reasoning skills in LLMs, including detailed planning, divergent thinking, intuitive judgment, timely reflection, internal thinking, and fact perception, etc. As CoT fine-tuning has advanced, LLMs have demonstrated substantial improvements in tasks such as mathematical reasoning and code generation. However, existing surveys about CoT fine-tuning primarily focus on technical aspects and overlook a systematic analysis from the perspective of human reasoning mechanisms. Given that the ultimate goal of CoT fine-tuning is to enable LLMs to reason like humans, it is crucial to investigate this technique through the lens of human cognition. To fill this gap, we present the first comprehensive survey of CoT fine-tuning grounded in human reasoning theory. Specifically, inspired by the well-known Six Thinking Hats framework, which systematically characterizes common human thinking modes using six metaphorical hats, we classify and examine CoT fine-tuning methods through this lens. Furthermore, building upon this theory, we outline potential directions for future research in CoT fine-tuning. In addition, we compile a comprehensive overview of existing datasets and model performances, and a real-time GitHub repository \footnote{https://github.com/AI-Chen/Awesome-CoT-Finetuning} that continuously tracks recent advances in this area is maintained. We hope this survey will serve as a valuable resource to inspire innovation and foster progress in this rapidly evolving field.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。