大语言模型中的元认知：基础、进展与机遇
Metacognition in LLMs: Foundations, Progress, and Opportunities
查看机构详情
- Yale University（耶鲁大学）
- University of California, Irvine（加利福尼亚大学欧文分校）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文全面概述大语言模型元认知知识现状，分析分类该领域，总结技术进展，涵盖测量评估方法、引发改进应用技术等，还讨论相关应用、问题挑战及未来方向，为该主题研究提供综述与指引。
中文摘要 AI 辅助
元认知是智能的基础组成部分，对有效学习、问题解决、决策、沟通等至关重要。近年来，它日益被视为强大、透明的人工智能系统的基石。尽管大语言模型在各种现实世界任务中取得了重大进展，但尚不清楚它们何时、如何或在多大程度上能展现或被赋予有效的元认知能力，以及如何利用这些能力提升人工智能系统的基本能力、可靠性和智能。本文首次全面概述了大语言模型元认知的知识现状，分析并分类了这一新兴领域，总结了近期技术进展，包括测量和评估大语言模型元认知能力的方法和基准、在大语言模型中引发、改进和应用元认知的技术，以及正在进行的研究的发现和启示。还讨论了应用、开放问题和挑战以及未来工作的有前景方向。旨在提供该主题的详细最新综述并激发有意义的研究和讨论。
英文摘要
Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems. Yet while LLMs have made significant progress across diverse real-world tasks, it is not yet clear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, nor how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first comprehensive overview of the current state of knowledge on metacognition for LLMs. We analyze and taxonomize the landscape of this emerging field and summarize recent technical advancements, including methods and benchmarks to measure and evaluate LLMs' metacognitive abilities, techniques to elicit, improve, and apply metacognition in LLMs, and findings and implications of ongoing research. We also discuss applications, open questions and challenges, and promising directions for future work. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful research and discussion. An organized list of papers can be found at https://github.com/yale-nlp/LLM-Metacognition.
