大语言模型中的涌现自我意识
Emergent Introspective Awareness in Large Language Models
AI总结：
研究发现大语言模型在特定条件下能识别自身内部状态，但其自我意识仍不成熟且依赖上下文。
摘要：
我们研究大型语言模型是否能够对其内部状态进行反思。仅通过对话难以回答这个问题，因为真正的反思无法与编造区分开来。为此，我们通过将已知概念的表示注入模型的激活中，并测量这些操纵对模型自我报告状态的影响来解决这一挑战。我们发现，在某些情况下，模型能够察觉注入概念的存在并准确识别它们。模型展示出一些回忆先前内部表示并将其与原始文本输入区分开的能力。令人惊讶的是，我们发现某些模型能够利用其回忆先前意图的能力来区分自己的输出与人工填充内容。在所有这些实验中，Claude Opus 4和4.1，我们测试的最强大模型，通常表现出最大的自我意识；然而，模型间的趋势复杂且对训练后策略敏感。最后，我们探讨模型是否能够显式地控制其内部表示，发现当被指示或激励去“思考”一个概念时，模型可以调节其激活状态。总体而言，我们的结果表明，当前的语言模型在某种程度上具备对自己内部状态的自我意识。我们强调，在当今的模型中，这种能力高度不可靠且依赖于上下文；然而，随着模型能力的进一步提升，它可能会继续发展。
英文摘要：
We investigate whether large language models can introspect on their internal states. It is difficult to answer this question through conversation alone, as genuine introspection cannot be distinguished from confabulations. Here, we address this challenge by injecting representations of known concepts into a model's activations, and measuring the influence of these manipulations on the model's self-reported states. We find that models can, in certain scenarios, notice the presence of injected concepts and accurately identify them. Models demonstrate some ability to recall prior internal representations and distinguish them from raw text inputs. Strikingly, we find that some models can use their ability to recall prior intentions in order to distinguish their own outputs from artificial prefills. In all these experiments, Claude Opus 4 and 4.1, the most capable models we tested, generally demonstrate the greatest introspective awareness; however, trends across models are complex and sensitive to post-training strategies. Finally, we explore whether models can explicitly control their internal representations, finding that models can modulate their activations when instructed or incentivized to "think about" a concept. Overall, our results indicate that current language models possess some functional introspective awareness of their own internal states. We stress that in today's models, this capacity is highly unreliable and context-dependent; however, it may continue to develop with further improvements to model capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
