面向大语言模型（LLM）集合的自我意识与自我调节元认知框架的实现
Implementation of a Metacognition Framework for Self-Awareness and Self-Regulation in Ensembles of LLMs
浏览论文内容
中文总结 AI 辅助
本文首次实现面向LLM集合的元认知框架，通过MSV量化自我意识，基于查询复杂度切换系统1/2处理，分配专门角色，经PoC演示证明其可行性。
中文摘要 AI 辅助
大语言模型（LLMs）普遍难以评估自身不确定性、检测知识冲突或识别问题是否超出自身专业范围，这些局限性不可避免地损害了LLMs的可靠性与可信度。本文中，我们首次实现了一种面向LLM集合的元认知框架，通过显式的监控与控制机制解决上述挑战。我们的系统计算元认知状态向量（MSV），该向量从认知心理学中衍生出五个维度，用于量化自我意识以进行监控：情绪反应、正确性评估、经验匹配、冲突信息、问题重要性。MSV值也为控制提供自我调节，根据查询复杂度自动在系统1（快速，单节点或多节点）与系统2（审慎，多节点）处理之间切换。对于系统2执行，图论算法根据MSV量化的元认知状态，将专门角色（领域专家、评论者、评估者、综合者、通才）分配给集合节点。我们的实现允许用户探索不同查询类型如何触发不同的处理模式。概念验证（PoC）演示通过示例展示了该框架，呈现了恰当的系统1/系统2路由，并通过实时雷达图和决策指标帮助可视化元认知过程。该PoC实现证明了在LLM系统中创建元认知自我意识与自我调节框架的可行性。
英文摘要
Large Language Models (LLMs) are notorious for struggling with assessing their own uncertainty, detecting knowledge conflicts, or recognizing when problems exceed their expertise; such limitations inevitably undermine reliability and trust in LLMs. In this paper, we present the first implementation of a metacognitive framework for ensembles of LLMs that addresses these challenges through explicit monitoring and control mechanisms. Our system computes a Metacognitive State Vector (MSV) quantifying self-awareness for monitoring across five dimensions derived from cognitive psychology: Emotional Response, Correctness Evaluation, Experiential Match, Conflicting Information, and Problem Importance. MSV values also provide self-regulation for control, automatically switching between System 1 (fast, single- or multi-node) and System 2 (deliberative, multi-node) processing based on query complexity. For System 2 execution, graph-theoretic algorithms control the assignment of specialized roles (Domain Expert, Critic, Evaluator, Synthesizer, and Generalist) to ensemble nodes according to their MSV-quantified metacognitive states. Our implementation allows users to explore how different query types trigger distinct processing modes. The Proof-of-Concept (PoC) demo showcases the framework with illustrative examples showing appropriate System 1/System 2 routing and helps visualize the metacognitive process via real-time radar charts and decision indicators. This PoC implementation demonstrates the feasibility of creating a framework for metacognitive self-awareness and self-regulation in LLM systems.
发表机构
- Fitchburg State University（菲奇堡州立大学）
- National University（美国国立大学）
- Worcester Polytechnic Institute（伍斯特理工学院）
机构由 AI 辅助整理，请以论文原文为准。
